CannonDesign’s Billie, An AI Companion Transforming Architectural Workflows

A Hello, disruptors. Welcome to this week's episode of Practice Disrupted. Artificial intelligence is rapidly transforming the way architecture firms operate. And today, we're diving into how one firm is leveraging AI to improve efficiency, collaboration, and access to knowledge. Canon Design, one of the nation's largest architecture and engineering firms, has developed Billy, a generative AI companion designed to streamline access to information and enhance workflows across the firm. But what does it take to build an AI tool like this from the ground up? And how does it balance automation with human expertise? To explore these questions, we're joined by two of the driving forces behind Billy's development. First, we have Emily Lauterer, Canon Design's national marketing operations leader. Emily specializes in optimizing marketing systems, leading the firm's marketing operations team and ensuring the tools that the marketing operations team use support business development.

A She played a key role in shaping Billy and believes in using technology to enhance, not replace human creativity. We're also joined by Joel Yao, director of digital products at Canon Design. With a background in design, entrepreneurship, and data strategy, Joel is passionate about creating digital solutions that solve meaningful problems. Before joining Canon Design, he founded Linear A, a firm dedicated to integrating data driven insights into the design process. Together, Emily and Joel will walk us through Billy's evolution from an AI hackathon experiment to a powerful internal tool, sharing insights on adoption, security, and the future of AI in architectural practice. Let's get started. Hi, Emily. Hi, Joel.

A Welcome to the podcast. I'm so excited to have you on here and talk about the new AI chatbot that Canon launched recently.

B Thanks, Evelyn. We're stoked to be here. Emily and I are really excited to to chat with you today.

C Hi. I'm happy to be here today. Thank you.

A So we usually open up with a little bit of an icebreaker of question to understand a little bit more about you, your personality, or anything you might be excited about coming up. Joel, do you wanna kick us off?

B Sure. I'm really excited because I just renewed my rock climbing membership for the first time in a year. I had a a knee injury that took me out for a while, and I'm really excited to get back in practice.

A That's amazing. Is it an indoor gym?

B It is. Yeah. I, go to Brooklyn Boulders here here in West Loop. So it's a shameless plug for them, but since this is heavily produced, I'm not too worried.

A Emily, any activities, hobbies?

C Sure. Yes. Let's see. I'm excited. I've got some fun projects coming up. So I am a sewer, so I'm currently working on finishing my eighteenth quilt. So I'm excited to wrap those up and give them to some family and friends.

A Amazing. Do you always give your quilts away?

C I just started this year, so I've actually done 18 quilts in a year. So I haven't been giving them away yet because I haven't been doing this for too long, but yes.

A So the reason why I wanted to bring you on is in 2024, I saw a press release come out of Canon talking about the creation of Billy and how architecture firms can use AI within their firms. So much of the conversation around AI is specifically about how do we use actually, I would say so much of the conversation, but also the consternation around AI is specifically how it's being integrated into our projects. So I I believe Billy goes beyond the project scope, but can you guys wanna kick it off and just tell us, you know, what what Billy is before we kind of go into the creation and what it does?

B It's like our assistant our enterprise AI assistant that it's really designed explicitly as, I don't know, to as a helper for for folks all across our firm, less so from a and Emily can talk to a lot about the, like, the inspiration behind it for sure. And I think, Emily, is there a particular description you like to use for for Billy when describing?

C Yeah. So it's our internal tool that leverages our advanced technologies like ChatGPT to search, retrieve, summarize any of our essential documents or data within our firm. So it pulls directly from our internal systems and has helped us change how we utilize that that knowledge and that data within Canon Design.

A For reference, can you help us, one, understand how big Canon Design is, but also what inspired the creation of of Billy ultimately?

C Yep. Yeah. So Canon Design is pretty big. We were we've got about 1,300 employees at the moment, and we do work across across the globe, really. So we're a pretty big firm with a lot of information, a lot of people, a lot of projects. We've got quite a bit going on. So I guess what kind of inspired the creation of Billy or where did the idea of Billy come from? So Billy was born out of an AI hackathon that we hosted last November where we brought together a small group of AI enthusiasts from across Canon Design. So from all of our different offices, our markets, our services, a handful of individuals were selected to participate in this hackathon.

C And we went into that hackathon with one goal, which was really to uncover opportunities to use AI across our firm. So before that, before the hackathon, we were given some homework. So everyone had to think through challenges, obstacles, things that either you or your colleagues face in our processes or our operations. And we wanted to identify, you know, things that were slowing us down, things that were making us less efficient. And one major theme that emerged from that was the difficulty of finding data. As I just mentioned, CannonDesign is a pretty large firm. We have a wealth of information. But kind of pinpointing that information exactly, where to find it or even how to find it can be very challenging and kind of time consuming.

C And that is very much where the idea of Billy came from. Billy is able to help us effortlessly search, retrieve, and summarize documents and our data within our firm. So it pulls directly from our internal systems. With Billy, instead of spending time searching through systems, we can simply, you know, ask and it will pull the information right to our fingertips. So it's very much transformed how we are able to use those processes, use those data, and give us a lot of time back in our days.

A Yeah. Joel, did you wanna elaborate at all on that? Or

B I think from, like, my perspective in technology and as a designer, it's really important for me that the time I'm spending, like, at work is value at a time. And a lot of time that is that design focused work that can the the noise around that can be pretty significant when you're dealing with a lot of different meetings, a lot of calls, a lot of things you have to research. And tools like Billy are an efficient way to get to the answer that you're looking for, kinda like a Google search for your own internal documents. With the goal there being that as designers, as builders, as constructors, as engineers, as folks that want to, like, really have that creative kind of mindset, that's where our flow state comes in. And Billy helps us stay there because we don't have to dedicate all this overhead time to just looking for something. And there's a lot of value to that from the design process knowing that at the back in the back of your mind, you'll be able to get the answers you need much more quickly than you may have before.

A Yeah. I was wondering if either of you could give us a very specific example. So one thing that, for instance, comes to mind is I did I worked at an architecture firm that did a lot of schools. Right? And you always have that project manager or that principal who's just like, remember that thing that, like, the acoustical ceiling that we specified in the the music room because they they had a great choir. We put something different in there. Can you go go look up that specification? And I'm like, what project was it? I don't remember exactly what project it was. It was some type of I think it was one of the high schools in this district. Just go go search all the specifications, essentially, or the the documents to find it.

A For me, I think you can literally kinda type that into Billy. Is that is that what you're getting at? And can it would re reference all those documents for me and kind of service that information.

C Yeah. Exactly. So you can you can ask Billy very specific questions. I come from the marketing background. So, you know, I get a lot of questions like, who was the project manager on x y z project? Where, you know, typically the process now is that someone has that question, they bring it to marketing, we go to the database, we search it, we find it, we send it back to them, where now they can just go directly to Billy and say, who is the project manager on XYZ project? And it's gonna pull that information. Billy will pull that information directly from our database.

A Amazing. So we know what inspired the creation of Billy. How did Canon create space and time for you guys and your team to create Billy? And I it sounded like the idea began out of some type of AI hackathon that you guys had.

C Yeah. So there was a a hackathon that we hosted in Chicago at the about about twelve months ago at this point. And, you know, we as a firm, we value innovation. We value things like that. And we just wanted to make sure that we understand that AI is gonna be a huge part of the industry moving forward, and we really wanted to put an emphasis on the technology and AI to make sure that we were staying in line with our competitors and also, you know, trying to get ahead of them. So we just value we value this, and we wanted to make sure we were putting time behind it to, you know, make progress.

A Yeah. Joel, is there anything you wanted to add? It was interesting because we were talking with you because we both hopped on this, recording early, and you were previously more on the consulting side. So you said this is, like, actually the first time in an architecture firm where you're on a tech product side. That's one a different mindset, but what does that say about kind of the evolution of how technology is going to influence architecture going forward?

B I think it's, like, technology and architecture have always been really intertwined. If we look through the history of design as well as, like, the the things that occur outside of architecture, like, how shelters changed, how, like, fuel sources have changed, how we do our work has changed since from drafting to AutoCAD to Revit to you know, now we're getting into, like, generative AI as a responsive nature to design. I think it's important that Canon is really prioritizing not necessarily the hype behind a lot of this, but really looking at the business value for these types of tools and not chasing something down necessarily in an academic sense, but in a really practical way knowing that, like, the quality of our work is reflected in how we critically look at these new tools. And so for someone as a design background that then spent seven or eight years as a consultant, as an entrepreneur, as seeing a lot of the problems and issues that our clients are facing on the front lines, now being internal to a company and now applying that consulting mindset to those same sort of problems has been really eye opening for me because Canon is using our, like, IT group, our technology team, our products team to really make and elevate the type of work we can do and what we can deliver our clients. I think it's really challenging as a consultant to provide advice to clients if you're not practicing it yourselves. And Canon does a great job of being able to say, hey. We're recommending this, or we're doing that because we're doing it ourselves. And I think we walk the talk a lot, and I I think that's really critical.

A So a lot of people when they're hearing about this, and Emily, you mentioned chat GPT, so I'm imagining you're using some type of API or something to pull it into to Billy. Obviously, there's privacy issues. So and you guys might have to help me explain to our audience what this is. But did you guys essentially create a rag with ChatGPT to implement Billy? And can you talk a little bit about that process to to our audience from somebody who is probably would have to be googling what is what is a rag right now?

B So the the way that I like to explain this is it's a it's like an enriched search. We're using without getting into the technicals as far as how it determines what you're looking for, there's there's a lot of things behind the scenes that are happening. But at the end goal, it's our data. We're not relying on, like, chat GPT's training data to give us an answer about ourselves. We're saying we're we're basically storing our own data in a very specific way, and then we're using to like, as you mentioned, API endpoints from large language models developed by OpenAI. One of the things that'll be coming is a mixture of experts, but I can which I can explain later, where we'll start to use different large language models so that we can get the best response. And but at the end of the day, we wanna ground our responses in a in the truth of sorts. So for us, that means we have to keep our data pretty clean.

B We actually go through a lot of work with, like, Emily and the teams in marketing and finance and HR to make sure the data that Billy has access to is curated, signed off on, and that we can restrict what it should be seeing. So it's not necessarily, like, looking at, like, Google searches and indexes the entire Internet. We're not necessarily doing that because there's a lot of things in there that some folks should be allowed to see and some folks shouldn't. And so we're really being very cautious with this in a way that balances the the convenience of these tools with the potential for oversharing of information.

A So the other thing that I think architects talk a lot about or not necessarily talk a lot about, but that I hear at least on the tech side with with RAG is this these idea of hallucinations. And, obviously so hallucinations being that since AI is generative, it's pulling all of that information, and it might be hallucinating back to you something that is partially true but not entirely true, which when writing specifications or other certain documents, you actually it needs to be more clear. So how do you guys manage that in the the building of Billy?

B Sure. I think we manage it in two we we can manage hallucinations through a few settings and things, like, in system prompts that we develop, which is our way of telling Billy about itself and what it can and can't do. And one of the ways that we restrict its ability to create answers that could be misleading is to say, if you don't know, don't create a response. And you can inject that type of logic into the back end of these models, and it helps it kind of provide some guardrails. Like, you're an you're an assistant, but you don't know everything. So don't make things up. And, like, the generative component, the hallucinations are as many as as much as people understand the risk there, they're really a feature and not a bug. Like, it's it's a bug of sorts, but this is still a generative model.

B It's doing kind of what it's supposed to do. And what we're trying to do is limit the occasions where that that can occur by grounding it in our own documents so that if there is a hallucination, it's coming from a starting point that's still our data. And we've not had we've had more folks ask, well, why can't it find something than folks question the results that it's providing back, which I think is a testament to how we start to optimize our settings and other things through the development process as well as the curation of our data on the back end.

A That's amazing. And it's amazing that now you're getting questions about, like, kind of the frustration of why isn't Billy delivering when it can on these things but not on these things. You know, obviously, one of the big sour notes on technology is that it's only as capable as well as it's implemented. Right? What did that look like? Testing it and then rolling it out to the bigger audience, and how is that going? Emily, are you getting less marketing requests?

C I think so. I mean, it feels like it's helping in a lot of ways. I think Billy Billy has some really great, you know, areas where it's exceeding and then some other areas where we still need to do a little bit more work.

B So I could talk a little bit about how how Billy's been utilized as far as the number of folks in the firm. We're seeing percentages that are pretty high on adoption. So Billy is not just being used by folks looking for marketing. It's being used pretty pretty globally. I think we've had over two thirds of the firm individually use Billy at some point since we've released since we've released the product. And we have about 30 25 to 30%, like, different regular users every week. So from an adoption perspective, it's it's pretty it's pretty high. And I think that's exciting for us because that gives us a lot of momentum to carry, like, improvements forward.

C I can give an example. I think open enrollment is a really great example of how billing has been used recently. So, you know, it's open enrollment season for a lot of different firms. And so like many other organizations, that's a time when HR is getting a lot of questions. They have a very high volume of questions, and they're only there's only a certain amount of people in that group. Right? And I think this was something that really helped new people get used to it. We put a lot of that information into Billy and made a lot of that accessible for Billy. So we really used that as a first line of defense.

C So people were able to ask Billy questions instead of going to HR. Of course, HR was there to follow-up if they, you know, if they if Billy didn't have the answer or whatever. But, you know, that was a great tool to get people starting to use it who haven't already too.

A Yeah. I didn't even think about that from an HR perspective. That's amazing.

B Yeah. It's it really shines on a lot of the admins, like and it's and I always like to tell folks, it's not replacing, like, a specific folks across all levels of the company to spend more time doing the things that they've expressed the desire to do, why they were hired to do what they do. And we've all been in corporate environments. We've all seen how there's they're like the noise becomes more additive over time. And I think tools like Billy really help us start to get back to maybe our core functions and the things that got us really excited in the first place. So for me, it's a it's an innovation that's welcome because, again, it gets back to the value added time of how we wanna spend our day.

A Joel, you alluded to something earlier. Was it market expertise and how that's going to be rolled into Billy?

B Yeah. It was kind of a technical description. You were describing a a RAG model, and that's what we're using. One of the things that we'll be implementing next year and I'm excited about is what's called a mixture of experts approach to machine learning and AI specifically, which will allow us to have some redundancy. For example, if OpenAI's API endpoint crashes or goes down, we have we need a backup plan. And there are plenty of other there are plenty of other large language models, foundational models out there from Anthropic, Google, others like open source ones that we will eventually start to incorporate into our products that will allow us to say, well, when someone asks Billy this question, go to this model to answer it versus treating one foundational model as the solution to everything. So it's a level of robustness that we're trying to build into our technical approach to these tools. But I think, again, Canon is behind in supporting and separates it from being a proof of concept to a really robust production ready, enterprise ready tool.

A Can you, just for our listeners, under help them understand what enterprise ready means?

B Sure. I think in the past, as a former employee of architecture companies and then my consultant and entrepreneur, there's always been a focus on getting a quick win and a proof of concept out there that works. And that's a great starting point. But as we as in the role that I'm in now of director of digital products, I'm really trying to take a life cycle view of these things and be able to say, just creating something that works on your laptop or works, like, kind of on the app or in an AI or in a browser or something, that's, like, step one. We really need to be focusing on how do we make it very secure, how do we make it really robust, how do we set it up in a way that the team behind us knows and understands how to make changes, it's scalable, it meets the demand for all of our users at once. The funny thing about Billy's rollout, it was announced by our CIO, Brooke, in Chicago at the State of the Firm, and we and we knew this was, like, our deadline. I I started in March, and this was July, I think, Emily, something around there.

C We launched in July.

B And that was the and so I started working backwards from July. Brooke's like, alright. We have to we need to deliver this in July. They wanted to do this this pretty grand rollout. And I was in the Chicago office where they were recording the state of the firm live. Our CEO, Brad, was talking, and Brooke started talking about, hey. We're gonna release Billy, and it went live at that moment. And I was sitting on my laptop after building out, like, the software engineering with my team watching the usage.

B And our usage spiked by, like, 50 x. Like, at once, you could kinda see this, like, splat line of our standard test users, and then all of a sudden, soon as Brooke mentioned it, it took off. And I was I was kinda terrified at that time because I was like, if it crashes now, this is, like, hugely problematic. Right? And, thankfully, we we had all we had everything set up in place for the enterprise ready to go. Right? And I think that's the that's the important component of it. We had a load we'd never tested before of users that we hadn't tested at scale, and it operated flawlessly.

A Amazing. So, Emily, you said you had a background in marketing. So for me, it sounds like you're a nontechnical person on a tech team.

C That's right. Yeah. Which which I

A I was a nontechnical person in, you know, working in technology and Slack. So I understand that. But can you help our, like, audience understand, like, the importance of your role on this team?

C Yeah. I mean, so so myself and a handful of others were on the conceptual team that came came up with the idea of Billy that came out of the hackathon. So that's, you know, that's kind of where it started. But then I think a lot of what Billy connects to in terms of the data so I'm in the role of national marketing operations operations leader, and I deal a lot with our data and our systems and have a deep understanding of all of that. So I think that's been really helpful to have on the team. Unfortunately, not all of our data is perfect. I'm sure most firms, deal with that. That was probably one of our biggest challenges with Billy as we started to roll this out.

C We we realized that it's only going to be as successful as the data that it's pulling from, which, you know, can be a challenge for sure. So I think that's definitely a place that was helpful to have nontechnical people on the project so that we understand this different type of data that it's pulling it's pulling into the system and and how to direct the team that was actually doing the work of developing Billy to the right sources and the right information so that we could get it going.

A Yeah. And let's talk a little bit about that hackathon. So, obviously, you mean, I imagine there were other things other than Billy that came to front and center. What was really compelling at least to whoever decided the the winners of the hackathon that, like, this is this is where we need to be investing our dollars right now.

C Yeah. So so a little behind it, there was a a hand everyone came up with an idea. We all pitched them. And then some of the ideas ended up being similar, so they were kind of formed into groups. And then we spent the day kind of developing our concepts, developing our ideas, and then presented them to a panel. And that's who that's who determined, you know, the the winner, so to speak, of the of the hackathon. I think they they looked at a lot of things. So was it feasible, like, was it helpful for everyone in the firm? Was it feasible? Is it something we can realistically do? We wanted to move, as Joel said, pretty quickly.

C So, I mean, that was in November, and we launched this in July. So we wanted to make sure it was something that we could do, and it was something that important was accessible to everyone. So that is something that Billy is something for everyone. It is not just for the architects. It is not just for our engineers. It is not just for the marketing team. It's something that anyone in the firm is able to use and utilize and get value out of. And that was something that I think was very important to the the committee that took a look at this and decided.

A What is behind the name, Billy?

C So the founder of Canyon Design was named William, William Canyon Senior. And I'm gonna have to fact check myself after this now. So when we sat down as a group, again, this all came together very quickly. We had about twelve hours to work on it, and so we were brainstorming names, and we came up with Billy. We like it's kind of gender neutral, and it was kind of a fun nod to our founder. So that is where the name came from.

A And then, I guess, you know, now that this has evolved evolved into our product and you still maintain your marketing title, like, how has your roles and responsibilities changed or evolved with the introduction in Billy and and its use cases

C too? Yeah. So when we first came up with Billy, there was a lot more involvement day to day. Right? We were trying to get everything sorted, get it all out there in in, the Canvasign universe and and move it forward. And now that it's launched, the concept team has taken a little bit of a step back. So we still meet with the developers and, you know, but it's it's less involved. It's more of an advisory capacity where we are continued to be asked questions about it or will be asked for help. But it's not as as hands on as it was during the development phase.

A I wanna jump a little bit back over to the technical side on Joel, if if you don't mind. So your job is to develop products. In my head, I'm like, if you are successful at doing what you're doing, inevitably, your team just continues to grow because that means you're gonna have to scale and support these things. How do you decide a product is working? And then how do you decide, no. We need to scrap this and move on to the next product?

B That's a great question. I think for determining how well a product is working and functioning, I really focus on user adoption and being able to say, hey. How many folks are using the tool? Does it track with our expectations or exceed them? If we thought, you know, 20% of the firm was gonna use Billy on a regular basis and we're at 30%, then I would say that's a measure of success. That from a kind of how quickly do we burn something down versus continuing to to fund it in that kind of way, it really for me, it comes back to business value and impact. So if we believe that there is a strong business value, and this is really for internal tools, like, there's a whole different strategy that I have for, like, external market. But for internal tools like Billy, it's really about understanding the business value and being able to say, if we're seeing a high adoption rate for a tool that we know is making folks' lives easier, then we want to continue supporting it. And we want to expand its capabilities. Billy right now is robust, but it's not done.

B It's it's like with anything in technology. It's an ever evolving space. There's always new features and and new ways of using these tools that can enhance our workflows that can make our make our lives a little bit easier when deployed properly and securely. And so we're constantly like, my my team is constantly, like, researching and staying on top of trends and and being able to say, this x, y, and z thing has reached a level of maturity that makes sense for us to start to test out internally. And then we work with folks like Emily and her team and other subject matter expertise across the firm to say, if we were to build something like this, would you use it, and would it make sense to to implement? I think there's a there's a challenge oftentimes that we want to and this is more just an industry wide challenge that I've seen in the past, which is we always wanna kind of chase the coolest thing, but it doesn't necessarily mean that's where the value is. And so I wanna be able to balance where the value is for us as a company, which is unique to how we operate, to our people, to all the things there with the technologies that are exist out there. Timing is really important. And being able to time the culture that we have with the technology available and roll it out in a way that is informed by our graphics team, which helped work on the UI for Billy.

B And, like, all of the things that Canon cares deeply about, the design, like, the experience, all of that, we have to balance all of it. And for me, adoption's the critical metric. The more that we see folks using the tools, the more that we know we've built something that's useful. And that means that we need to continue to support it. And if there are if we see certain groups that aren't using it as much as others, we go to those groups and ask, hey. Could we improve Billy by including x, y, and z data or having it do a different thing? And if they say yes, then we start to test it out. And we work directly with them as the folks that know that information, the data, and their workflows the best.

A Another one of a topic that is always top of mind when it comes to AI and AI integration into workflows and processes, mostly on the design side, is just like the ethics, the use of the AI, what dataset is it drawing from, you know, and is there any biases that are built inevitably have been built into that dataset just from historical context? So what were kind of the ethical considerations that you guys were exploring when Billy came together? What did those conversations entail?

B I think this is kind of why I see the the idea of a a mixture of experts, like, mixing multiple models together, especially in the future state when we start exploring more generative design components, things like Adobe Firefly, which they have legally said, like, we will we will go to court for you if someone says that we're using your content because they're that confident in their training data and their ability to say we're not, you know, we're not stealing our customers' data. Like, I would look at that and say, well, that's the API endpoint. I would potentially wanna explore more for, like, a generative component if we decide to integrate that into Billy. And these are all decisions that have not been made. These are just kind of the way that I would approach the ethics behind it and how we've engaged our legal and our compliance teams especially to say what we're very transparent with users that have Billy. There's a splash screen that shows up that says, hey. This is the type of information that Billy is sourcing from and your expectations for privacy and all of that. We're pretty transparent about that through and this is like an internal tool, and we're still splash screening it up when you use it.

B So I think for me, it's about being very clear as much as we can about the the sources of the information. And when it comes to the the open, like, large language models and the foundational models like chatgpt.com versus Billy, this is why that RAG component comes in, is we're providing it a ground truth that really tries to limit its creativity to just using our data and our voice so that it's not misrepresenting Canon design even internally. And that is not something that any one team has complete and full control over. We're just making a best practice effort to to keep the generative component focused on where the value is, which is search summary and, like, then guiding folks in the right direction for things versus trying to repurpose a voice or a specific person. That's where I see a lot of the the ethical challenges. I wouldn't ask Billy to write a proposal like Emily would. I'll put it that way.

C I was just gonna add out outside of Billy, we also have a a group that's working on working with our legal team, our compliance team about AI in the workplace. So for tools outside of Billy that involve AI, we're we're connecting with them to set standards, policy that that, you know, address the how we're using AI, how we're using client data, how we're to make sure all of that is done ethically and making sure we're not you know, there's no bias and things like that as well.

A How do you guys identify additional use cases for Billy? So, like, Evelyn, you gave kind of the HR and open enrollment. Like, that was kind of an an annual thing that I'm sure popped up on the radar, and somebody must have said this would be a good idea to populate Billy. How do you guys identify those use cases and then work with Joel's team for implementation?

C I think they kind of come up naturally. So we have Billy we have Billy attached to a lot of different data sources and turn including our own intranet. And so I think a lot of people have started to realize, you know, I think, Joel, you can correct me if I'm wrong, but I think the HR team kind of realized, like, oh, this is a great resource. We're gonna be asked 50 questions an hour about open enrollment, and how can we if we give Billy this information, if we make it accessible, we can use that as a first line of defense. I think another example is like onboarding. You know, when people are onboarding, we're telling them to use that for questions like, how do I figure out how much PTO I have? Where do I how do I submit my time sheet? You know, simple questions like that. And so we're starting to introduce that to new individuals when they start, and it becomes part of their just daily process to use it. They're learning it right from the beginning of their their time here.

C And then I think a lot of it just comes up up naturally as people start to use it. They realize the opportunity that that it creates for their information to become more accessible to others and to get their information out there. So it's really kind of naturally happened.

B It's kinda similar to how Google started as a product and now it's a verb. And we're not taking the same, like, I'm by no means saying that we've been at the next Google. That's not the the point of that conversation. But the idea is we changed our process to Emily's point, and and now HR onboarding content, other content includes and go to Billy and look for these things and screenshots of the responses that you can kind of expect. So it helps folks from day one. And as a recent day one employee, this would have been a really helpful tool if it had been live when I started in March. Like, that I mean, I I use Billy regularly. And the other nice the other nice side of it, to Emily's point, is when we do see consistency in the questions, because that's one of the things that we're tracking and storing and being able to understand, well, what are folks asking Billy? And are they we're seeing the same question 400 times.

B And Billy's response is, you know, we do a thumbs up, thumbs down, kind of like, does it work, does it not? We can start to understand where the gaps are. And those gaps could be data. Those gaps could be model driven. But it gives us a really great starting point to say, if I'm seeing a hundred different people ask the same question, I should probably make sure that my response is correct. And if Billy's always nailing the response, great. We'll look at the next one down. So it's a really it's a way for us to perform analytics against the type of questions folks are asking. So, again, business value there.

B Like, what's the fastest path to the right answer? And Billy for us is is a way to get to the end result more quickly than we may have otherwise.

A So I'm I'm gonna spin this back around because I I wanted to talk about, like, how these type of tools can support the operational side of things. But, you know, is Billy at all being used to influence design or support the design process in any way at at this point in time? Do you upload any design documentation via specifications or kind of plans or anything like that, to even to even support the even project research aspect of it?

C It's not. Right? It's not being used for any rendering, any, like, visual concepts, anything like that. It really is a tool that helps with our process in terms of time saving and streamlining those kind of more repetitive tasks or routine things so that that high value work dry that helps drive innovation. That's what people really have time for.

B We have plans for next year to start integrating Billy as a plug in into some of our design tools as a way for us to be able to to do exactly what you were describing. If I'm working in Revit and we have a plug in that talks that Billy that's like a Billy plug in for Revit as an example, and I could say, do we have specifications around this type of thing? Do we have this information? Again, thinking about it more from a search recall like the RAG model type to be able to give folks in the environment they're currently working in access to Billy without having to step outside of that environment. So I look at it as a way to start to integrate Billy as a widget, as a helper, as a bot in different areas, and that can be from our service desk ticketing to a Revit add in to a Rhino plug in to get that kind of experience moving. And to your point earlier about ethics, as we have clearer guidelines around ethics and the generative component to creative work, we can start to open up different endpoints, and we'll already have the infrastructure in place to do so. For so for me, it's starting to to lay the groundwork for more creative work if we decide to do that in the future, but we're not rushing into something that could be ethically fraught just because it's cool. Like, we wanna, again, focus really on where that business value lies and and where it aligns to, like, our company ethos and our and our mentality in general.

A What other projects Emily, if you were to go back into the the hackathon again, what opportunities what other opportunities do you see potentially coming out out of AI, or would you like to see a cannon pursue next?

C There were I mean, there were a lot of great ideas that came out of the AI hackathon. Another one that I liked was using AI to create client profiles, almost like a deep fake of our clients so that we can kind of present to them in certain ways and get their feedback on our pursuits or our submissions or things like that.

B I think one of the things I'm excited about is going to be moving from the the large language model that we currently know on the generative side to the agents, like, agentic models. So being able to actually kick off a series of tasks and those things being a bit more independent of the user. So being able to ask Billy to go do something and then have an action taken and performed and the results provided back. Again, thinking about, like, how we work and the type of work that we do and being able to streamline that a bit, getting back to that value at a time, being able to say, how can I spend the most amount of time adding value to my clients, adding value to the industry, adding value to my firm and my company? This is a way for me to do that. And I'm excited to explore a lot of the the kind of the next step, which is the agent based workflows.

A Yeah. I was gonna say back to, like, specification and, like, products or reviews and submissions. I I feel like there's an opportunity there to say, if you're getting RFIs or submittals via email to do that intake and have a draft response sitting out, and then you just need to kind of check off a few more boxes and hit send.

B Yeah. Like, being able to summarize, like, a massive amount of questions that you just received and say, give me the high points and be able to and then reference where those high points came from so that it's not hallucinating back things that, you know, it's just like, well, I think this makes the most sense. Like, being able to, again, have that ground truth and still improve your workflow, have the convenience of the tool without sacrificing the quality of the work or injecting misinformation into the process is really critical.

A So I think, you know, kind of as just as a closing question. Obviously, I feel like you two are kind of early adopters in this space in using AI, especially as it comes to your workflows. And then, Joel, you're obviously creating a product around it. What does the conversation look like for your peers in this space who are more hesitant to pursue the technology? What would a takeaway be for them?

C From a marketing side, it's kind of a a, like, you you gotta adapt or you're gonna be left behind a little bit. I think, you know, I I think we don't have to be fearful of it. There's a lot of ways to use AI that are going to benefit your day to day workflows and processes that isn't going to take away the creative aspects of your jobs or it'll really just give you more time to focus on the parts that you really like. So don't be afraid to explore it a little bit and give it a go because there's a even if you start using it in small ways, it can create a lot of efficiencies in your in your day to day.

A Joel, any thoughts?

B I think the advice I would give, it's it's twofold. It's one, understand the value of your own data. Like, data isn't waste. Data can be waste. It can be junk. It can be it can be noisy, but there's a lot of real value in the data that we're all generating. And tools like this help us achieve more value out of the data that we have. So just better practices around making sure the most up to date documents are in the right place.

B Like, just little things that we can build good habits around when it comes to data quality will enable these tools to be that much more efficient, that much more value added for each firm. So that's one side of it is the is the data quality piece. And the other side is this use the right tool for the right task. Understand even at a high level what these tools are designed for, what's a feature, and what's a bug. And don't try to conflate the two. I think that a lot of times folks are rightfully upset about the things that large language models do, and it's it helps to understand that they're doing as designed. Like, we're trying to put I I like to tell people it's you know, you're giving a you're giving a Ferrari to a 14 year old and then expecting them not to speed. Like, these are designed to convince you of the thing that you're talking about and respond in a way that is convincing like a teenager would to try to get the keys to the Ferrari.

B And so for me, it's just helping folks understand even at a high level, the strategy behind these tools should always involve your legal and compliance teams. It should always involve your data teams, especially with HR and other things where there can be some really sensitive information out there, and just know the fundamental risks and limitations of these tools and be able to use the right tool for the right task. It's a great example of why we don't use Billy for certain generative components of it, but we may use other tools. And to Emily's point, we have an entire team looking at that. So it's trying to balance, again, convenience with the technology. It can generate a lot of convenience, but it can be equally as cumbersome if used in the wrong way.