Building the Decision Engine for Agentic Marketing, Advertising Week New York, October 7, 2026. Speakers: Annie Leanse (FOX One), Olivia Kory (Haus), and Jon Humphrey (Rokt). Moderator: James Hercher (AdExchanger). This transcript has been lightly edited for clarity and length.
James: Thanks for joining. I’ll let the three of you quickly introduce yourselves, who you are and what you’re doing here, and then I’ll get into it.
Annie: I lead subscriber acquisition marketing for FOX One.
Olivia: Hi, I’m Olivia Kory. I’m chief marketing and strategy officer at Haus.
Jon: And I’m Jon Humphrey, and I lead the advertising business at Rokt.
James: Great. So, a good one: measurement, which everyone has figured out. We can all just get off stage. It’s an interesting panel, too, because we’ve got FOX One, which everyone knows is a media company, but here you’re the advertiser. And I think of Rokt as bringing demand, but representing the media side of the supply chain here. So let’s start with FOX One. Tell us quickly about your measurement and incrementality approach, and what the past year or two has looked like for you.
Annie: Well, for those who don’t know, FOX One is FOX’s streaming service. So if it’s on FOX, it’s on FOX One. We launched just over a year ago, so I’d say we’ve had a very interesting year of measurement, because we started with no data. We work with Haus now, but you need about six months, probably a year, of good data to actually have something to start measuring against. So a lot of the first six months, which a year ago was NFL and college football, was my team just guessing. We’ve all come from streaming, and we were hoping we were making the right moves.
Then we had the World Cup over the summer, and that’s when we really started to leverage measurement and think about the big swings we wanted to address first. A lot of it was channel level, always focusing on incrementality. People are going to sign up for FOX One because they want to watch their Rams on Sundays. But how can we ensure that my media dollars are actually bringing in incremental subscribers? We do that by using Haus.
James: One good place to jump off from there: How does implementing an incrementality program affect paid media? People often think of measurement as post-campaign work. How does it come back to, okay, this actually changed where we spend our money?
Annie: Where you think you’re driving incrementality, you’re not always doing it. And it forces me to slow down, right? When you’re using last click, you can very easily, day to day, hour to hour, see where subscriptions are coming in from, which is exciting. A lot of times you want to start acting on it right away because you’re seeing efficiencies. I’ve been trained well by Olivia and the Haus team in the incrementality framework: You have to slow down, let the experiments run, and really use the data and those learnings to make any shifts. Because over-correcting too early is where it gets dangerous.
James: How much of your job, Olivia, is just telling people, “Don’t. Just don’t. Ignore the dopamine hit of whatever that last click was”?
Olivia: We recently rolled out daily incrementality, and it’s been both the greatest and the worst feature release of my life. We’re publishing results daily, but I have to tell people, “Don’t make decisions off of three days of data. Please don’t make decisions.”
Annie: I don’t, but I look every day.
Olivia: It’s tricky. I’m not going to ask for a show of hands, but I think incrementality is a buzzword that’s not as widely understood as people think. The way we approach incrementality at Haus is like randomized control trials in healthcare. It’s the classic scientific method. We turned off marketing in these regions. We left it on in these regions, and this was the true business lift that we saw as a result.
That can be painful. When you run experiments on your marketing program, where I tend to see brands get stuck is, number one, the results that come back look so different from attribution. Sometimes people say, “I’ve been looking at the same Google Analytics report for 10 years, and you’re telling me the thing I’ve been taking to my C-suite for a decade isn’t as effective as we thought?” And I say, “Yes.” So I spend a lot of my day coaching brands on how to have that conversation, because it’s very tricky.
The other place folks get stuck is that an incrementality test is a moment in time. A very seasonal business with something like the World Cup is going to see much different incrementality during the World Cup than in an off period. What we say is that it’s like taking pictures. Yes, it’s a snapshot in time, but that doesn’t mean you stop taking pictures. You have to keep taking pictures to fill the album.
The last piece, which we’re probably going to talk about a lot on this panel, is that measurement for measurement’s sake does not add value to an organization. You actually need to action on those results, and that’s how you drive business outcomes. We see a lot of teams standing still with the results. They’re opining over the data, potentially arguing over it. I spend most of my day asking, “What are you going to do about it? This is the result. If you don’t take action, you shouldn’t expect to see ROI on your testing program.”
James: Is that frustrating? Why not take that next step yourselves? As a measurement vendor, your job is to push, prod, and inform, but you’re not the advertiser.
Olivia: This has been an interesting part of our journey at Haus. We founded the company in 2021, born from the ashes of iOS 14, to rethink what measurement should look like. In the beginning, I think we thought our role in the ecosystem was to be the objective third party. Here’s the data. Have a nice day. What we’re learning is that people really want our point of view. We sit across this giant database of experiments. We have hundreds of customers, we’ve seen tens of thousands of experiments, and customers want to use that to improve their business. So we’ve had to take a much stronger point of view on marketing and what brands should do.
Annie’s piloting this with us, but we now surface recommendations to our customers. We call the product Architect. It’s a pun on Haus. It answers: What is the next best action I can take to make more money for my business today? We’re surfacing things like, move this much budget out of Google, move this much budget into Rokt. And we’re seeing that a lot of those recommendations are actually getting implemented.
James: This one’s for Jon. We were talking before about the marketer who shoots from the hip. There’s a lot of cost to measurement: the holdout group, paying a measurement vendor, paying to repeat the test. I’ve heard nightmare scenarios where a marketing team ran a 10% holdout in China and totally messed up the finance team’s forecast. So is there pushback? What would you say to the marketer who says, “Forget that. I’m going with my gut”?
Jon: The first thing I’ll say is I appreciate those marketers as much as I appreciate the ones who measure everything. I think of measurement and incrementality as truth-seeking. You’re constantly on this journey to figure out what’s true. Your instinct will often tell you one thing. I instinctively think I should put money here, I should invest in Rokt or Meta or Google or TikTok, and so on. Usually those instincts are based on prior experiments you’ve run or companies you’ve worked at. When you spin up a new service, and FOX One’s a great example, you just don’t have the data to make those decisions.
So a lot of this is truth-seeking. Is this incremental? Does it drive great LTV? There are all these gates you go through before you make a decision. When I step back, though, truth-seeking gets you to one point: Now I have a lot of these answers. There’s still a huge element of instinct and gut feel in marketing decisions. There’s also a huge element of trust. Do I trust that this channel is going to make the best decisions on my behalf with the dollars I provide? I don’t think you’ll ever reach a perfect world where you measure everything and only measurement can tell you the answer. Marketers will always have a perspective, even in the age of AI.
That said, I think you can get to the truth faster. A lot of this is about speeding up the journey to the answer, and then constantly retesting. The best marketers I’ve ever worked with can do both, and Annie’s a great example. You can have an instinct about where to invest, but you measure constantly. It’s not a point in time. It’s something you do over and over again. The ones that only measure things and only believe what the measurement says will probably actually get themselves tripped up. The inverse is also true. The ones that only fly by the seat of their pants will probably get themselves in a little bit of trouble, especially when you’re managing tens of millions of dollars of marketing budget. So I think the answer lies somewhere in between. There are some interesting products out there that get you to the truth much faster and help you make that decision, while still balancing your own instinct.
James: On the question of speed: Maybe part of why some people are still stuck on last click is that it feels like a speedboat. You can just move. What does it take to get comfortable and say, I have enough data and I feel confident making this decision? There’s this daily incrementality mindset, but I’m sure some people are skeptical of it. Annie, maybe you can start.
Annie: I feel like we can’t move fast enough, but we also have to police ourselves. It’s that dichotomy. One thing that got me so excited about Architect is that there are certain channels where my team or my agency just can’t move fast enough, and paid search is a great example. FOX One is all about live. We have FOX Sports, which is live sports, and we have FOX News. When there’s a huge upset halfway through the fourth quarter of a game, or there’s breaking news because something’s going down in Washington, my team tries to react as fast as possible with campaign changes. But with AI, with something like Architect, that can happen with one click of a button. We can say yes, push more to paid search, increase our spend on social, and pull it back from a more expensive CPM placement. Taking the same recommendations we’d probably reach on our own with a little more time, but having AI in the middle to speed us up: That’s the dream.
James: Are you speaking as an advertiser there, or as FOX One the broadcaster? Say a baseball game goes to extra innings. Wouldn’t viewers be more interested now?
Annie: Both. We’re always trying to drive urgency and tune-in, so if we can capitalize on moments like extra innings, why not? Those people are incremental. They didn’t necessarily come in at the beginning of the game, but if I can convince them with my media to come in for that moment, I just need to act faster.
Jon: That’s one of the cool things about the independent measurement story. Rokt is an advertising channel. We work with thousands of advertisers, and one of the things the team and I are really passionate about is leaning into independent measurement. Haus is a really good example. I have a saying, and I’ve said it to Annie before: Advertising is a bit of may the best channel win, right? There’s so many places that advertisers can place money. I can only grade my own homework so much. Independent measurement actually helps you as an advertising channel tell your story.
Olivia and I were talking about this before the panel. She asked me, “How do you think about Rokt? Is it mid-funnel? Lower funnel?” And I joked, “I think it’s a lower funnel, highly incremental channel, and I have no idea how to message that on my own.” But when you let third-party independent measurement in to help you and your advertisers measure it, all of a sudden they can help you tell a story you didn’t even know you had. Advertisers like FOX One, and thousands of others, start to understand your channel a lot better and a lot faster.
So I think of speed more like velocity. Speed is fantastic, but on its own you might end up running in the wrong direction. Speed and direction is velocity. When you combine the two, you run in the right direction, and I think Architect’s a good example of that. As an advertising channel, there’s a lot of storytelling you have to do to show people why they should invest with you when they could put 90% of their performance marketing dollars into Google and Meta. A lot of brands still do that. So when you’re an up-and-coming ad channel, working closely with third-party measurement providers helps you learn more about your own channel than you knew before, and helps you tell that story. I think that’s an exciting development in the industry.
James: Let’s say you’re a media company, whether that’s a broadcaster or a retailer, and the incrementality scores come back and they’re not good. Maybe the marketer still suspects the channel is useful. You don’t want to just take your grade and go home. How do you get better and fit into the incrementality framework? Can you respond with something other than, “Sorry, we’ll be better”?
Jon: I know a bad way to respond: Get mad at the measurement provider. As an advertising channel, you’re going to run hundreds, if not thousands, of incrementality tests across your advertisers in a given year. Some are going to be absolutely fantastic, some are going to be horrible, and they’re going to surprise you. The cool thing about incrementality measurement is that you can improve. You can change the way you’re targeting, the way you’re doing attribution, the way you’re doing creative. When you build really good trust with advertisers, you can work through those things together and figure out how to make the channel more incremental for whatever their objectives are. So the way not to respond is to get really annoyed and push measurement providers away. That’s not going to help you in the long run. I’m sure you’ve seen both sides of that coin.
Olivia: We have a running joke at Haus about what we call the firing squad meeting with certain ad platforms. They’ll show up with nine people against our one poor measurement strategist, trying to invalidate the results. It’s a very bad look, and I’d advise the platforms in the room not to do that. For a brand just starting to look at incrementality and testing a new channel, why would you expect incrementality to be amazing out of the gate? You’re not optimized for incrementality yet. We have some great success stories where a brand launches a test on YouTube and gets a one back in incremental ROAS. They optimize their creative, their targeting, and their campaign structure, and by the end of the year, they’re at a four. That’s what success looks like. The best teams understand how a quote-unquote bad result is actually a very important learning for the business. It’s nothing for anybody to be ashamed of. Why would you expect incrementality to be great if you’d never looked at it before? It’s a new way of looking at things.
James: Annie, going back to speed, can I ask you a quick question? You said it’s sometimes hard for your team to respond as quickly as news and culture unfold. What are the biggest blockers? Is it getting creative fast enough when a sports team is trending? Is it getting direction to the media agency to allocate budget? Where do you feel stuck?
Annie: My head of creative is in the room, so that’s not the problem at all. It’s really about getting the signal and knowing how to react to it. We all get so many push notifications, and mine are mainly sports-driven. When breaking news comes in, there’s no way for us to know whether news outlets will cover it for the next 30 minutes and then move on, or whether it will last and pick up steam. AI has the ability to track that. I guess I could hire someone to watch Google Trends all day, and when something crosses a certain threshold, we flip the switch. Right now, we can figure out the creative, and we can figure out how to activate in-platform. It’s really the signal that this is worth spending more on, because there’s always a risk when you’re heavying up. We’re going to take a gamble either way, but the human aspect of seeing the signal and then responding is too slow right now, and we could miss that moment and then also not know when to come back down.
I keep going back to paid search because in a single day, we could go up and down so much based on news alone, but we can’t do that for performance purposes yet.
James: And with that paid search signal, if you’re looking at the platform, how much confidence can you have in a real-time, in-platform signal that isn’t based on incrementality?
Olivia: This is where I think we’re struggling as a measurement industry, by fighting complexity with more complexity. The whole idea of triangulation sounds great in theory, and it is totally messed up in practice. It’s not working, and marketers don’t want to do it. By triangulation, I mean the three legs of experiments, MMM, and attribution. People are getting really tripped up by this in market and in their jobs. In terms of speed, MMM has historically been extremely slow. More modern MMMs have sped it up, so maybe you get a weekly or monthly macro allocation view. Then you still need attribution. As much as we talk about how bad attribution is, you still need it for the daily and hourly calls. If we have good signal, and experiments calibrating both the macro-level MMM and the micro-level attribution models, we get one step closer.
But what we’ve learned is that it’s a lot to look at. It’s overwhelming. You’ve got MMM, attribution, and experiments, which means three dashboards. That’s where AI is a real opportunity. Should a human actually be trying to reconcile all of these data points? It probably doesn’t make sense, and that’s why we’re bullish on AI automating a lot of the data analytics and the synthesis of these signals, to get you closer to acting on them without needing human supervision.
James: That’s the agentic side of it. You can imagine both a broadcaster’s agentic setup and an advertiser’s recognizing the value of a breaking news moment or a big moment in a game. I often hear programmatic companies, particularly in live sports, describe this as an opportunity to bid on that moment and extract value the publisher isn’t claiming. Is agentic a rising tide that lifts all boats, or does one side lose out on an opportunity that’s always existed?
Jon: We were joking about this upstairs before the panel. A few days ago, a client sent me an incrementality study that was run two and a half years ago and said, “This is what we’re using to make decisions.” I had a laugh. But when you step back, it’s easy for us to sit up here and talk about speed and all the cool measurement strategies we can use. There are a ton of organizations with marketing budgets in the hundreds of millions, if not billions, of dollars using data from years ago. One of the things I think about is how to remove the friction, even in really large, complex organizations, so they can measure faster and make decisions.
It’s a journey, though. It is not easy to align an entire organization around an MMM that one team owns and an attribution model that a different team owns in a different country, organization, or line of business. That’s really, really common. You have nimble organizations like FOX One that make these decisions really fast, but not every organization operates like that. So I think a lot about how to help an advertiser take the next step, wherever they are in their measurement life cycle. I can assure you that using a study from a few years ago probably isn’t the best thing to be doing. But how do you help them take one first step, like bringing the MMM and the attribution model one step closer together? The more you can help them do that, especially as an ad channel, the smarter decisions they’ll make. Everybody’s at a very different point on this journey.
Annie: I’ll add that the role of awareness media, or brand media, is at such an interesting point. Whenever people bring up the funnel, I always say the funnel doesn’t exist. At the same time, my product’s only a year old. We have a lot of user confusion, because people know what FOX is, but maybe not FOX One. Sometimes the performance marketer in me says we need to heavy up on certain channels. Or, I know Rokt is really incremental, but maybe it’s not growing my brand the same way a CTV takeover is. So it’s about understanding where your product and brand are in their life cycle, to know which decisions matter most. Sometimes having high exposure with little impact on incrementality today is the right move, because I know that putting in the effort now will help later. I’ll say any new product should have the World Cup on its service in the first year, because it brings a lot of people in. But moving forward, I still have to make sure people know what FOX One is, so when they see me on Rokt, they’re in the mindset of, “Oh, I know what that is, and I’m ready to…”
James: That’s the judgment aspect, right? There has to be an overlay of judgment. How close are we to agentic judgment? What are you comfortable with now, and do you think that will be different a year from now?
Olivia: I think you need to build that into the system. We work with a customer whose CEO loves billboards and wants to see the brand on a billboard on Sunset in LA on the way to work. I like them too. We just need to build that into the system. There are certain settings and configurations that are immovable. Another one is the CEO who needs to see their brand on a branded search page. It’s optics, and the objective may be separate from subscriber acquisition. So we built a configuration settings page for exactly that: Here are the immovable objects we won’t touch, regardless of what performance says.
The next thing that brings judgment in is business context. This is the number one objection we hear: “You don’t know about my promotions. You don’t know about my new product launches. You don’t know about my content slate.” And it’s like: Well, sure, but we could. Just feed that to us. So we’ve made it very easy to send us everything you have, and I think good MMMs do this. Data ingestion becomes key. Make it easy for customers to share that with a partner. The more business context, the better.
The last piece, where we see people struggle to feel secure in the judgment, is explainability. I think this is what people dislike about Google’s and Meta’s algorithms. It’s “Trust us. We’ve got it. We have this great machine learning model.” I know Meta has a great machine learning model, but it would be really cool to see the signals it uses to decide how to allocate spend across creatives. If you can pop the hood a little bit and give marketers a little bit of insight into why we’re recommending the thing that we are, I think that goes a long way. So we’re bringing judgment in through business context, through custom configurations for how your marketing organization works, and through more explainability.
Jon: I would be shocked, and maybe even a little disappointed, if we were sitting at Advertising Week next year and media buying wasn’t agentic. I think that would be a massive missed opportunity. If you do the measurement now, you can move faster, improve outcomes, grow the economics of your business, and reinvest that money. It’s such a powerful thing. Ad channels benefit, and customers benefit. I’m pretty sure we’ll make that shift. What I’m curious about is whether everybody makes it together, or whether certain segments of the industry and certain types of advertisers adopt it faster and outrun everybody else. I have a feeling that might be the case. I’m excited to see where we land, but as an advertising channel, I think it’s in our best interest, and everybody else’s, for media buying to get there.
James: What does that mean for the humble media buyer or growth marketer? Are you going to be AI-ified?
Annie: I don’t think so. To Olivia’s point, a lot of my conversations with Olivia and her team are about the variables that impact our media. I can upload my NFL schedule, but I can’t predict breaking news, or someone getting injured. Every business deals with so many variables, and I think that will always have to be human. And going back to the person who likes billboards, there are things you do as a business for a reason. But touching on what Jon said, I tell my team that if we’re actually lowering our investment year over year but seeing the same outcomes, then that’s how I know I’m putting my money in the right places. I know that’s no platform’s dream, but if I can keep getting more efficient and know I’m driving more incrementality and pushing every dollar further, then I think we’re all here for it.
Olivia: I’m a CMO, so I run marketing at Haus, among other things, and I would love it if you could automate media buying for me. I have so much else to do. I’m constantly surprised at how tedious a lot of marketing operations still are today. It’s shocking. I think we should all be trying to automate this. It’s scary when you’re spending as much money as you are. Adding an extra zero to a campaign by mistake could put you out of business tomorrow. But what a dream it would be for some of this more tedious work to be automated away.
James: I think we’re out of time, so thank you all. We could keep going, but we’ll let you go. Thanks, everyone.
