[A] Podcast
00:00
audio controls 10 10
audio controls 10 10
 
00:00

Evolving Information Experiences with Language Models and RAG

[A] Podcast #35

backforward audio controls 10 10
backforward audio controls 10 10

Listen:

Listen on Apple Listen on Spotify

Interview With Bill Rogers

Join Bill Rogers, founder and CEO of ai12z as he discusses the evolution of content management systems (CMS) and the impact of artificial intelligence (AI) on the industry. He shares insights from his experience in building platforms like Ektron and Orbita, and highlights the shift towards decoupled architectures and the use of large language models (LLMs) in content management. Rogers explains how the combination of structured and unstructured data, along with AI agents, can create personalized and efficient customer experiences. He also emphasizes the importance of curiosity and experimentation in driving innovation in the field.

hexagon

Bio

Bill Rogers is a visionary entrepreneur with a deep technologist background in AI and digital technologies. Recognized for significantly influencing the evolution of online experiences, Bill founded Ektron and served as its CEO. Under his leadership, Ektron emerged as a pioneering SaaS web content management platform, serving thousands of organizations globally. After Bill sold Ektron to Accel KKR, it merged with Episerver and became part of Optimizely. Bill then co-founded and led Orbita as its CEO, driving innovation in advanced conversational AI. Beyond these startups, Bill co-founded several other ventures and has had an expansive career in digital signal processing and robotics engineering.

Bill holds a Bachelor of Science in Electrical Engineering from Boston University.
 

Resources

Follow Bill Rogers on social media: More about ai12z:
"these AI assistants are really reasoning engines. And by enabling them ... they get to a goal and know what steps it needs to perform."

Transcript

Cruce Saunders
Welcome to Towards a Smarter World. This is your host, Cruce Saunders, and I'm really pleased today to be joined by a longtime friend in the industry, somebody that has built across content management, artificial intelligence, and now leads the way in a new organization called ai12z. It's Bill Rogers, founder and CEO of ai12z, a visionary entrepreneur who's got a deep background in AI, digital content technologies, and chatbots. Early in his career, Bill actually founded one of the most important companies in content management, Ektron, which became a significant player in really setting the pace for what digital content management could be. He also was the CEO that helped it to grow over many years until its sale. Into what eventually became Optimizely. Optimizely, of course, nobody needs an introduction to as one of the leading platforms today across enterprise CMS. And Bill has also after that then founded an organization that was driving conversational AI with Orbita and Orbita innovated various content technologies all around Alexa and similar voice skills. So he's been working in interactive content. And now with the advent of large language models, Bill's busy, brilliant mind has been at work again, creating a new platform in ai12z. Bill, welcome to the show.

Bill Rogers 
Thank you very much. for having me Cruce on the show. Pleasure.

Cruce Saunders 
Well, I'd love to just start back with the early days, and then let's move in and closer to AI. We were all kind of involved in these early days of web content management, and Ektron was an early innovator on that, creating the basis for the WYSIWYG content editors that became kind of fundamental to content management as the internet was evolving and relying more and more on complex content objects. Could you tell a little bit of that early story and give a sense of how you got it going originally?

Bill Rogers
Sure.

So there's really two parts of the business. The first part of the business, we created WYSIWYG editor. A lot of people will know it from EWebEditor Pro, which was integrated into most of the other CMS systems out in the marketplace. But basically what it was designed to do was to enable non-technical people to be able to author content, kind of a word-like editor. Then we expanded from there to create a product, a .NET content management system. With the goal really is to, how do you make it easy for people to create and manage content and deploy it out onto the web?

Cruce Saunders
Well, great. And that basic simple idea, of course, became an entire suite of software. A lot of platforms have evolved since then. We've been involved with a few of them. But I know we're dealing with a complex environment now of over 4,000 CMSs.

I'm curious what you see from those early days of CMS development to today and all the headless and composable revolution and all the SaaS delivered offerings that are happening. What's driving the change and how do you see the content management system market evolving?

Bill Rogers
So I think over time what you've seen was that organizations would build a solution. And the reason why other vendors came into the marketplace is that they've seen other problems and they want to address them. So obviously when we were building a content management system like CMS 400, our goal was to take and deploy, make it easy for people to publish content. But as time went on, the web became much more larger, the requirements for getting data and bringing it to the web meant performance issues. And so, you you see headless coming along where they want to have like an API first type strategy. They want to be able to not only get content from one CMS system, but multiple different systems. And so, vendors created technologies to solve problems. And so you see that really expanding out with the headless space today, especially around things like Omnichannel delivery. So that's another reason why people are looking at some headless solutions to deliver an Omnichannel solution out into the marketplace.

Cruce Saunders 
Yeah, content APIs back from the early days when it was exposing a little bit of content as a service. Content as a service became a doorway to the real -time content APIs that were very faceted and able to really decouple presentation completely from the back end. And so now we have all these decoupled architectures that are really dominating a lot of the conversation and many, seems like many of the entrenched, you know, I guess some people are calling them legacy platforms or monolithic platforms, but platforms that there's more tightly coupled environments in beginning are moving more towards API based, you know, interactions with the front end. And a lot of that seems like it's with this universal CMS movement, all moving in the same general direction, which is making content decoupled as reusable objects, moving around in lots of different ways between systems, different content platforms, different devices, presentations, and channels. Does that sound like what you've seen? And how do you see it moving and progressing as a market from here?

Bill Rogers 
So I think what happens to the legacy content management systems, they see that problems are being solved with headless and they look at it and say, well, maybe we should be solving that problem as well. And they're sort of not forced into doing it, but they actually start to understand.

why are organizations migrating to these headless solutions and should we be doing that as well or should we just be partnering with headless solutions? And so that's the dynamics of the marketplace and supply and demand kind of drive things going forward. And so I think, you run into scenarios that where people that are on the authoring side and how do they kind of experience what the experience will be for their solution? So the headless players are looking at those kinds of, how do we address those kinds of issues where the legacy systems are trying to figure out how do they solve delivery type problems in the marketplace and provide that kind of a solution as well.

Cruce Saunders
That's great. Do you see this decoupled revolution continuing?

Bill Rogers
I do, and I think it's because...

You know, I think developers are our API first and Headless gives them that kind of ability to take advantage of it. And I think that, you know, they see that the advantages of performance with, with getting just the right information that they need and delivering it with just that information, as opposed to getting this huge packet of information. And depending upon which channel, some of it is just useless data and slows things down.

And so they like the fact that they can query the right information and get just what they want to go deliver the solution that they want to deliver.

Cruce Saunders 
Great. Yeah, nimble, fast, composable, responsive experiences that are created on demand by channel based on what the customer is saying. That's always been the goal. We've just been getting there in all kinds of ways. And one of the ways was chat bots and getting that real-time content. So your next chapter after you sold Ektron was Orbita. And Orbita is all about real-time content in this new thing of digital assistance. Can you tell us a little bit more about that, chapter?

Bill Rogers
Yeah, Orbita, so we were making both solutions that dealt with things like Alexa or also dealing with chatbots. But this is where you can give a highly experience to an individual that's visiting the website, because we could dynamically change what's the next thing that we say to the user. So we chose health care because it was a big problem. And so you can imagine when someone interacts on a healthcare website and they want to find a doctor or a location where they have an issue and they want to say, I have a stomach ache, which doctor should I see? Well, stomach ache doesn't really work well if you went to a search box and just try to search for it because you won't find any results. you're not connected between when you do the search on the site and how do you actually find a doctor where what we were trying to do is solve those types of workflows and make it so that when our patient interacted with a healthcare website that they could get to an end goal as quickly as possible.

Cruce Saunders
That's great. And that contextual information delivery. So a customer says something and then we respond with some content is a pretty technical solution. There's a lot of moving parts. Can you describe some of the moving parts to get the right answer to the patient?

Bill Rogers 
Well, during those times, there was some additional challenges where those challenges were related to things like we were using things like intent models. We were creating a knowledge base where we would have to curate the content. And so we were working with a cancer institute, for example, and we could create about, I think we created about 800 questions and answers. And so you'd have to create an intent model to deal with each one of those facts. And so that was a lot of work, which has completely changed with the way that you would address that kind of a problem today, which is just use the content that they already have and allow a RAG system to be able to interact with that content in a way that there's no intent models that are required to go go build it.

Cruce Saunders
That's really, really interesting. Yeah. So, so we'll definitely get into what a RAG is for those listening at home and trying to get terms. you know, we, back in the day, we're working with healthcare organizations and content models that, that then work with real time content APIs. I remember we had a joint client and a large healthcare provider with.

lots of different content types, know, diseases and symptoms and treatment plans and various kind, everything from recipes to other kinds of content types. And we modeled all of that in this very intricate way in order to create an API that could then be consumed by the Orbita platform and then generate all of the chat responses in Alexa.

So now that architecture is evolving. And I'm really curious in what ways it's evolving from having highly structured content types to moving towards large language models, which actually really are used to training on unstructured data sets and just give it web pages. so I'm curious how this relationship to content structure is evolving. And it's a bit of a beefy question to start out with, but I'd love to hear your thoughts on it.

Bill Rogers
So it's interesting to, when you think about a virtual assistant, an AI assistant that's on a website, there's really two ways that you're going to bring data back to the user. One way is through this RAG type question and answer. And RAG is for retrieval augmentation, right? Or retrieval augmentation generation. And so what ends up happening with RAG is that you ingest all of this data into your RAG system and kind of core to the RAG system is a vector database. So the vector database is almost like a search index, except for instead of storing what you would say key words, you're storing meaning, which has a real huge benefit to it. So when you ask a question, the RAG system will say, well, go get me similar documents to what that person's asking for and feed this to the large language model so that it'll analyze it and summarize it into an answer. Why the meaning is really important is that you can ask the question in any language because the language is you're transforming that into something called embeddings, ask the question to the vector database, you get the results, and you're telling the large language model to answer the question in the language that the person asked for. So even though you've only ingested English, you can answer the question in Chinese or Arabic or Korean. It doesn't matter. And the LLMs are actually very good at language translation.

So how the RAG system keeps the, what we, what people would call hallucinations from not being a problem is that you're telling the large language model, only answer the question based on the context that we give you. Don't make, don't make things up. So if you said, what is the capital of France? It's going to say, I don't have an answer to that question because there's nothing in the and what if ingested with that would have told it that answer. The LLM certainly knows the answer, but it doesn't give you that answer. And that in healthcare is really important because you want to have it answer questions based on their content and not on what it thinks it should create the answer to.

Cruce Saunders
Yeah, so now we're not assembling answers as much as generating answers out of vectors. And so I'm curious what is ultimately the role of content structure?

Bill Rogers
So you mentioned that RAG is just one agent. Another agent could be something different. So for example, let's say that you're interacting with a hotel and the person asks the question, what are the menu items that don't have nut allergies? So the problem, that's a very bad thing to ask a RAG system.

Because a RAG system, so imagine that you have 200 menu items and 50 of them don't have, they have not had allergies. And so what will happen is that when in a RAG system, you really only get to pass a few documents to your alleged language model. So in that scenario, you're calling an agent.

And the agent is actually doing a search to return menu items that don't have nut allergies. And so that structure is actually important to the agent. So now it returns items that can be summarized by the LLM to the user.

Cruce Saunders
OK, great. Yeah, so you're seeing a world in which the combination of the ingested unstructured content that creates vectors also works with the structuring process and the sort data preparation process to give the LLMs the cleanest relationships within content structure by inference also the semantics, the taxonomies of terms and relationships of terminology.

Bill Rogers
So let me describe one more thing because it's pretty exciting to be honest with you. So now imagine you're asking a question, what are all the restaurants that are near you? So now you're just saying, I'm gonna call an agent and it goes out and it gets a hundred restaurants that are near you. It's all structured data. That structured data actually goes into the LLM. But when you say, let's say restaurants that have Italian food, that structured data that just went to the LLM. Now the context windows of these LLMs are getting huge. Open AI, it's 128K tokens, about 100,000 words that you can ingest into it for doing this question and answering. Google is now up to, I think, to me. It's large, the processing speed isn't slowed down that much from what you put into the LLM. It's really slowed down on the generation. So if I take this structured data, put it into the LLM and say, show me Italian restaurants, it's going to take that structured data and it'll only show you Italian restaurants, which can include multimedia. So like in the structured data, you have it linked to the image, linked to video. It'll, it will then spit out. The content to be showed in the chatbot of just that data. then, and so that again, in this scenario, you get the best of both worlds. You're using structured data coming into an LLM. The LLM is intelligent enough to sort it down. You don't even have to put pre-filtering on your, on the server that's going to get the results. It can do the filtering within it and give you results. So there's some amazing capabilities that when people think about LLMs, think just you can only work with unstructured data, but no, you can work with structured data really, really well.

Cruce Saunders
So is it a fair statement to say structure in, signal out? Is that fair?

Bill Rogers
So what's important about what you just said is that if you give it good structure, the LLM is really good at taking that good structure and using it and giving you better results. like with the really important point there, if I said go out and find restaurants and I took, I put into the RAG system, each restaurant into the RAG.

And I said, go find Italian restaurants. If I have 50 Italian restaurants in there or 20 Italian restaurants, I'm only going to pass four documents to the LLM. now, so basically again, the answer of four documents where if I pass it this, the JSON a 50 of, know, like 200 restaurants, which is not that much content over to the LLM that would include things like the images, the description, the title of the restaurant, address, phone number. If you think about it, it's not that much data. If you give it to that, it's going to know which ones of these are Italian restaurants, and that's when it's going to return back to the user.

Cruce Saunders
Great. OK. Well, this is great because over time, we've been getting from big chunky, WYSIWYG blocks tied tightly to the presentation being delivered on demand of a web page to real time answers being composed out of our many content assets, structured and unstructured, in more and more harmonious ways. Higher and higher relevancy in real time across multiple channels and to your point, languages. This is huge. I mean, to me, this feels like we are absolutely in this revolutionary moment in content history. I'm curious what just sort of big picture you're most excited about and what gives you the most pause for concern, those to sort of big, big picture things.

Bill Rogers 
Yeah. the big, what I'm most excited about is that, is that these AI assistants are really reasoning engines. And by enabling these agents, the reasoning engine says, I know what I need to go do. So for example, let's go back to the example of using the hotel website. If, the reasoning engine actually is being passed a system prompt that says, what's the address of the hotel? What's the address of the airport that's near you? And you go to the hotel website and you say, can you give me directions to the airport? It will know to call the Google Maps agent. And now all of a sudden, you're JSON, structured JSON from Google's Maps API into the LLM, which has the driving directions and it's rendered just like it would be on in the Google phone. It also creates the link that you can click on and Google Maps will open with the exact same driving instructions and it creates the map and shows you the line from the start to finish of what that is. So what's so exciting is that you don't need to do all this programming that you would use to do. So if you have an agent that calls this JSON, the LLM just gets the JSON and it knows what to do with it. And what's, nice about that is that means that every agency in the world can take any website in the world and say, do you want to have a workflow? What's your backend systems? I need a JSON API that represents the data that I need. And you can now create a workflow really, really easily. Just go over and create an agent that pulls that JSON out of it. And then it's showing, you're starting to see some incredible solutions that used to require you to spend a million dollars to build a solution. now, within a day, you have a RAG system and then you start to...

Remember, we used to talk in the CMS world about having a maturity curve. And what steps should you do in the maturity curve to go do things? Probably get a rank system running up first. Then say, OK, now how can I take advantage of structured data from the CMS so I can get better results to that? Third thing is, let's start enabling agents that are either out of the box, like Google Maps or, you know, Google web scraping through Google or whatever number, like an agent, like Salesforce agent. So that when someone comes and you're a company that sells products and somebody says, status of the bug? It knows, it needs to send you an HTML form to understand who you are and get the ID that we've passed you when you opened the bug up so that it can call the Salesforce agent to go look up the status and bring those results back. But again, for you to go program that, you're just interacting with an agent that exists to your Salesforce connector and you have a form that collects the data and you get the results. So it's amazing, it's just how fast you can do things that would have been like a year's worth of development is now, it's...

Cruce Saunders
Amazing.

Bill Rogers
I guess for me, working with agencies, the most exciting things is agencies are going to be doing much more strategy. How can we create workflows that are going to create efficiencies in your business?

Cruce Saunders
It really opens up a whole new gamut of possibilities that never existed before. the implication is also the interfaces will change. We're used to loading web pages and searching web websites. How is the UX going to change now that things are moving towards intelligent agents interacting with each other? @mrcruce What is that like for the customer?

Bill Rogers
So there's going to be, so you'll see these AI assistants all throughout the web page. And you see it inside of systems. CMS vendors are using these agents to automate tasks that are in the CMS system to handle a publishing event, how they're willing to do the deal with things like SEO. All of this stuff is creating efficiencies for people to do it.

When you look at the delivery side, when you actually have one of these agents on your website, now you're talking about having web controls that have different purposes. And those web controls are going to have controls that make the experience really efficient for what you're trying to do. So you might have an experience where you say, okay, I'm going to go call out and interact with that hotel website. And I want all of the restaurants, the experience might be that you're throwing it in a carousel as opposed to just thinking like how you see something like Chat GPT where it just gives you the results and it grows and it grows and it grows. That's one way of delivering the experience. But there's times when you're going to bypass the LLM because the agent get the data and you're telling it, I just want the structured data to be shown to my client. I don't want the LLM to go through a processing step. want you to just put the data that I got and put it in. So you're going to see that, but you're also going to see it in commerce sites. You can imagine that you get some structured data and you have something that you say, buy now. And so you're going to have in your controls, this ability to call JavaScript to do things like add it to the shopping cart so that it's you. And so people can do ask questions around it. Another kind of an example that people are getting excited about is how we compare products with each other. So you can bring products in the carousel and check, check, check, check which ones that you like and say compare. And now you're bringing those products back to the LLM. The LLM will do a RAG search in each one of the products and then do a comparison and show you a comparison. And as you know, most websites that have products, comparisons are real hard to go do. Not hard for an LLM to actually...

Analyze the information and give you a comparison. And a comparison might be like an example on the hotel website, you may tell you, here's three wedding events, know, places that you can have like a wedding. And you might say, I want to compare these event places. And I want to make sure that I'm going to have 80 guests. put in, like the person will put in some of the requirements, it will analyze these different wedding events and actually come back and say, well, these two don't work because you have too many people. And it will figure that out. Whereas, know, think of the program that you would have done to just go handle that. And they can just do it naturally, which is wonderful.

Cruce Saunders
My gosh, I love these cases because it's illustrating for the listener, know, some really kind of idea sparking ways of understanding what's now possible. And I'd love to kind of do some more case batting around if you've had some really interesting experiences now, because you've been implementing these RAG systems for companies.

And can you tell us a little bit about search and how search is changing in light of, RAG architecture? Cause clearly, the old keyword density, it's not enough anymore. So what, what, what is, what is search doing?

Bill Rogers 
And so you'll see that there'll be web controls that are for search and chat bot and knowledge boxes. So two things you can think about in search. When you're using like a search control, it's going and it's going to query the the RAG architecture and get back an answer. What we see is amazing results where they compare it to their own search, which will often return no results. And they see that not only do they get the results, but it gives them information and calls to actions on next step. So that's another important piece when you're using these LLMs. It's not just getting an answer, but actually having like call to action buttons. Like the example of if I went over to a hotel site and I said, what are the activities you have? And it gives you a list of activities like golf and tennis and things like this. But next each activity says learn more. And you click learn more about golf and it'll tell you about the facility golf that's there. Then it'll have schedule a tea time. So if you click on schedule a tea time, you're actually calling an agent that looks out and starts an input form to say, what's your name? What's your first name, last name? And then it's going to ask, what date do you want to pick? And so when you pick a date, it's dynamically changing the time slots that are available for the user to go pick from. then they pick their time, and they go to the next step in the dynamic form. So again, this is, you have this ability to say what should happen client side and what should happen server side, because all of that collecting the form, calling the dynamic, pick a day, pick a time, that's just calling a call to your agent to say, go get me time slots for this day, go get me time slots for this day. So you, but you're not going back to the co-pilot and using the LLM to go analyze each step of that way.

So it's a balance of having the right kind of what happens on the client side and what happens on the surface side to make all of that kind of stuff.

Cruce Saunders
Interesting. Yeah, so there's also new challenges for the development teams in sorting out how to best allocate effort and order of operations. I'd love to hear your thoughts over the years. You've worked with many, developers. So if you're giving advice to developers now on where to focus in this new era, How can people be refining their skills for the development community out there?

Bill Rogers
I think it's, I think honestly it starts at the executive level. I think the executive level of organizations need to say this new channel has emerged called AI. And so everyone in the organization needs to think about how do you, how do we be curious about that? And what kind of problems should we think about that it might be able to solve?

Then they have to say, okay, there has to be some sort of learning phase. How does the internal organization start to learn about what can we do? How can we deal with this? honestly, this is, think, where agencies are going to play a big role into all of these organizations, because they're going to come and say, we have this skill set that can look at your processes and help you transform those processes into something that can...

That can go and work. And I do think that, you know, anyone that as an engineer, what does it mean for me? mean, I think really being curious about AI, doesn't take long to get up to speed. You can, you can go to places where you can even do testing, even going to chat, GPT and testing things out that there.

You'd be surprised of like how you can get chat GPT to actually, do things where it's you, you're in a back and forth conversation, which you can get it to do. And ultimately what that's what you're trying to do a lot with these reasoning engines is get it to know that it needs to get to a goal and what steps does it need to perform to get that? When should I call agents? Sometimes I need to call agents, do something and then call an agent. Like you might say, go, mean, somebody says, I want to sign up for golf. You're calling the agent to go collect data. Then you're analyzing that response. Then you have to call an email agent to notify them that it's been booked so that they have a record of it. And so that kind of workflow process, the reasoning engine can do that with just you going over to the system prompting and encouraging it sometimes.

This is the goal of what you need to go do when somebody signs up for an activity. want you to email them the response and make sure that the results of the activity is registered into whatever the backend system, whether it's Salesforce or some other back system that needs to know that information.

Cruce Saunders
Yeah. So it sounds like we, the executive level, start with curiosity and inspiring curiosity within the teams about how AI can be applied and how it changes the possibility set within customer experiences and within workflows and processes across the organization, get people experimenting and testing it out. And as an engineer, just crack open Chat GPT or Claude or one of the other interfaces and start working directly not necessarily with the API, just start working with prompts and creating a dialogue and then experimenting from there and then getting deeper towards code. Great. Also, for entrepreneurs who are looking to innovate in these evolving fields of content management and AI, you've certainly...

It just hit lots and lots of success in all these fields. I'd love to help our audience who's looking to follow in your footsteps and help to contribute to this ecosystem. What are your thoughts on how to create success in these fields?

Bill Rogers
Well, I think from when I look at the history of a couple of companies that have been at what sort of formed that was, are we really solving a problem that somebody cares about? So the editor was something that was pretty obvious. People wanted to be in a WYSIWYG environment.

In content management, we had something called Page Builder. You can drag it around. We were trying to help marketers build things that didn't require developers to handhold every step of the way in doing that. In Orbita, we were looking at how do we create something that's going to make it easy for patients to navigate healthcare. So I think what you're...

You have to do if you're looking at how can I build something? So AI just opens up so many capabilities of things that can happen. think this concept of building agents ultimately means that people are going to build their own agents that have some AI that might do something. whether, know, data, we've always said in the content management, content is king, but data is king now that we have all these back end systems. So what kind of back end systems as an entrepreneur could you enable and create actionable insights? I look at possibilities like, you know, even in the CMS system.

We have all of these logs of collecting data and honestly, there's not much that we do with them, but those logs have so much actual insight into it. Could you build AI that's going to analyze logs to give marketers more actual insights on what is actually happening with people that are interacting with their solutions? There's all sorts of those kinds of things that could be exploited in the future. And frankly, everything has APIs today, so that means that you could go build something and extend it onto something and move forward with something as an idea.

Cruce Saunders
Yeah, the pace of innovation is absolutely awe-inspiring right now. I'm blown away by how quickly things are changing @mrcruce and inspired by all the possibilities that all of these systems are creating together. Like you said, it's all APIs interacting, and now intelligent agents will be the layer that helps us operate services on top of these APIs. Our interfaces are going to totally change and evolve along with it.

So it's definitely exciting times. Thank you so much for joining today. If people are interested in following your work, where should they tune in?

Bill Rogers
ai12z.com and then from our LinkedIn channel is a good way to to go see that as well. But you'll find that on ai12z.com.

Cruce Saunders 
Great, that'll be in the show notes. And again, this has been an amazing conversation. I always enjoy talking about where we're at with you. You've got so much perspective on where we've been, where we're at and where we're going, and you're leading the way. So thank you, Bill, appreciate it and have a great day.

Bill Rogers
Thank you, you too, Cruce. Thank you, bye bye

Cruce Saunders
Thanks everyone.
 

Highlighted Quotes