February 24, 2021

Gathering Data for Zero Trust


Max Saltonstall (@maxsaltonstall), Developer Advocate @ Google Cloud


Topics covered:

Do you have something cool to share? Some questions? Let us know:


>> PEACOCK: Hi there. Welcome to the Cloud Security Podcast. Thanks for joining us today. Your hosts here are myself, Tim Peacock, product manager here in Cloud. I look after our threat detection products and Anton Chuvakin, a reformed and tamed former analyst and member of our cloud security team here at Google. You'll find this podcast wherever podcasts are distributed and on our website, if and when we finally do launch it. In the meanwhile, you can follow us on Twitter at Our guest today is Max Saltonstall, a develop advocate here at Google Cloud. Max, thank you so much for joining us today. We are stoked to have you on the podcast and I think we will get right into it. So today's topic is Zero Trust or BeyondCorp. Max, what are these terms? What is zero trust? What is BeyondCorp?

>> SALTONSTALL: So this goes back about 12 years or so when Google had to do a fundamental reimagining of how we do security inside our company. And the effort was called BeyondCorp cause it was an effort to move beyond a corporate network. We had to go passed the idea that you're either inside the corporate network, unless you can get access to the things and we presume that you are safe, or you're outside and you have no access at all. Like when you're working from home or from Starbucks. And we realized this was a problem for a host of different reasons and so we started changing things. More recently, Zero Trust has become the meme, the term around a lot of these efforts and it really just means I don't implicitly trust my network. And I'm a big fan of BeyondCorp. I'm a big fan of Zero Trust. I do think they both get thrown around too broadly and can be very confusing when people often will say them but mean different things. So it's great to define our terms here and dig into it a little bit.

>> PEACOCK: So this is really the move away from enterprises being a crunchy exterior with a soft chewy inside.

>> SALTONSTALL: Yes. No more M&M's, no more eggs. It's about abandoning perimeter security. It's about contextual access. It's about testing each and every time somebody tries to get to a resource, whether that's a website, a database, a VM and checking, "Do I actually trust Max enough to let him in right now?"

>> CHUVAKIN: I think the broad definition is something to risk because frankly, in my experience I've encountered people who say, "Zero Trust means you cannot trust anything." And frankly, this is false because you obviously need to trust the Zero Trust system itself. You need to trust the identity. You need to trust endpoint stay data. So let's go have a discussion about what else must be the use to ultimately trust it for Zero Trust system to work?

>> SALTONSTALL: Yeah, you have to build a chain of trust. Completely. And with BeyondCorp and the way we've decided to implement Zero Trust at Google, you do have to start somewhere. So often that means you're starting with a trust in some of the information you have about your people because you've assembled it yourself having hired them. And some information you have about their computers, which you're getting from agents. A frequent challenge I get there is, "Well what if the agent gets compromised then what do I trust?" You don't wanna place all your trust in just one agent or one data source but what we do is we gathering information about the devices that employees are using and we have information already about the employees, right. I know Max, he's in New York, he works in engineering and so that gives me a way to design a system that can grant or deny access based on that data. And I trust that data because I know here it came from, I know how I got it and I have other data sources to corroborate it.

>> CHUVAKIN: So is it useful to talk about the route of trust in a system like this? Like what's the real route of trust? What must be trusted otherwise the whole thing collapses?

>> SALTONSTALL: I mean you trust that I'm not an alien in human form, feeding information to my over lords in Orbit Pine the Moon. But you have to trust some level of agent based data about your devices. Right, I'm not in everybody's house right now, as we work from home, checking on their computer and I have to trust something that I'm getting back. Whether that's from Chrome's Endpoint Verification, whether it's from CrowdStrike or whatever it is, I need that device identity and device metadata. And I have to trust my HR system, right. However I'm tracking all these people and the groupings. Tim says he's in product management. Do I really trust that that's true? I mean I could go check but if someone had access to the way we track employees, had that had been compromised, that's definitely gonna derail my Zero Trust system for sure.

>> CHUVAKIN: This was substantially that line identities are been accurate and correct. So that means multi factor identifications needed, the actual identity should be attached to the right people and people should be grouped in the right way. To me this is probably the second big part of what I would call route of trust, the Zero Trust is identities. Not just device data.

>> SALTONSTALL: Exactly. And the reason I don't just trust Tim to say who he is is that he's also authenticated with a strong second factor so that it's very hard for someone to spoof or phish or otherwise impersonate his identity. And that's an important part of the whole chain of trust in the way BeyondCorp or Zero Trust implementation should work.

>> PEACOCK: So it sounds like we really got three pieces of trust here, not Zero Trust. We've got identity trust, we've got multi factor trust and we've got device attestation trust. So not zero trust.

>> SALTONSTALL: Yeah and I think identity is something I would maybe break out into a couple pieces. Is this actually Max or someone pretending to be him? And then there's is Max in the right groupings? Right. Did he get the right entitlements that he inherited by the factor of being in cloud or being an engineer or being in developer relations? So yes. And I think it's good to point out the second factor that I am definitely trusting the strong authentication of choice and without that, this whole system falls apart. Right. If you have weak or punishable authentication mechanisms, please don't even try to do Zero Trust. You have even bigger fish to fry and you need to do that right now.

>> CHUVAKIN: This is an excellent point and I feel like it needs to stand out in the podcast. It's a good one. Okay, so let's look at this as a project, the company, the organization they do. So in your experience Max, what's the first step in a Zero Trust implementation that increases the chance of its success? Like what can an organization do in the very beginning to make sure that their journey to Zero Trust is a successful and be not so painful and whatever else?

>> SALTONSTALL: I usually give people three parallel first steps. Which is a terrible answer but I'm gonna be honest with you. So one of them is what we just talked about. You have to have that strong authentication. If you're not doing strong multi factor of, don't do the rest of this. It's pointless. Just don't. Number two, in parallel one B, whatever you wanna call it is you have to understand your devices in some way. Sometimes clients come to me and they wanna do BeyondCorp or Zero Trust and they have this wide open BYOD policy and people are using just whatever laptops they happen to have at home and I don't know anything about the device that's signing in to my corporate information, I don't wanna trust it at all and so I'm not gonna give that device any trust. And I have to some way to either gather information about the devices, put some agents on there and roll them in some device management something. And then the third part is picking a really small use case. And the most likely success built on early momentum. If they come and they wanna boil the ocean right from the get go, it's not gonna work. They just gonna get stalled, they gonna get stuck, they gonna be mired in approvals or they gonna just be reading documentation for three straight months. It's not gonna be happy. So I always encourage people to look for a really focused used case. And that means an application that will be not so painful to move over and a small group of people who rely on that application. Right if you gonna take down your payroll processing system that every single person relies on, please don't do that at step one. But if you wanna use some small web app that maybe just this team of 10 or 20 people uses once a week or two, that's a great place to start because you can recover from any catastrophe more easily and you can learn. You can get the comfort level for you and your IT and your security teams in a lower stakes environment before you move on to higher stakes migrations.

>> PEACOCK: That's a very agile approach to this then. You start small, have early wins and then work your way up from there.

>> SALTONSTALL: From a change management perspective, yes. From a infrastructure perspective you really do have all the pieces in place. Right. I wouldn't say start with an easy application before you've done your strong authentication. No. Please don't do that. Right. Go fry that fish. So I would the device data and the person data, so the authentication part and device health part, those have to come first. Then you take an agile approach to larger and larger used cases or scopes, working incrementally, establishing some early wins, building the skills, building momentum. And it's also helpful to convince stakeholders. Sometimes there's some skepticism from inside and being able to point to a group that says, "Hey, I'm actually happier now that we moved over because I don't need to rustle with those VPN software cause I can work from my phone or my laptop. Because I could work from that hotel or from that flight or train ride." And you open up more possibilities for people to do flexible remote work in ways they couldn't always have done beforehand.

>> CHUVAKIN: So you're suggesting that a targeted approach where you select some users and you select some apps is what we should do to succeed?

>> SALTONSTALL: I think once you've got the device and the people data really down, then picking just one or two applications to start with that impact a small number of people at your company is the best place to begin.

>> CHUVAKIN: Okay so as far as the more practical aspect of a project, promise to share some details on the data you need to gather before you can do Zero Trust? What kind of data is that? Presumably you would need asset data, presumably you mentioned you need strong identity.

>> SALTONSTALL: Yeah the other part that we haven't talked about yet is the security policies. So you do need to the people, right. What groupings they're in. You need to know how they're authenticating and how much do you trust those methods of authentication. Cause that might not always be the same. You need to know what devices they're using because you can't really grant trust to a session unless you know what it's running on. And then you need to have a conversation, usually between the application or service owners and the security or IT teams, around what are the tiers of trust that we're gonna create and which of these destinations requires which tier, right. You wanna go looks at bugs in our current production product. Okay that requires the highest tier of trust. You wanna go see what vacation is coming up next month in our corporate calendar. Okay that requires the lowest level of trust. And maybe there's some gradations in between. That's a conversation between humans. It's a policy conversation, it's a risk conversation and it's a good one to have before you start implementing because everyone goes about this slightly differently. The error I frequently see is they actually spend way more effort on defining some super fine grand system. You don't need more than a handful of gradations of trust. But you do need to map each of your applications to one of those gradations and figure out what's the minimum that Max needs in order to go look at this codebase or that database.

>> PEACOCK: That makes a ton of sense and I can imagine that being both an exercise in bike shedding and an exercise in a lot of spreadsheets. So bike sheds and spreadsheets, people say Zero Trust is hard. Would you agree? Is it hard? What are the places where people get stuck? What's so hard about it?

>> SALTONSTALL: I think it's hard for some organizations. It depends what they're coming from and what they're trying to get to. As we mentioned, Zero Trust tends to mean a lot of different things to a lot of different people. But I think the starting points are not hard. Understanding your people are something every company has to do cause you know those people wanna get paid. And if you don't know what job they have, you don't know what to pay them. So we're already part of the way along that journey. Understanding your devices is something again that most companies are doing. There's a couple that have policies that make me scared at night but for the most part, they are doing it. And the tricky part there is just the integration, right. How do I pull the data I have about people's laptops and move it into that security decision-making system? Whether it's BeyondCorp or some other one. Connecting those darts is the hard part. And it's hard just cause they don't know how to do it. Sometimes they also doing it for laptops and desktops but they're not doing it or they doing a different thing for mobile devices and so you have this weird mismatch. I know a lot more about laptops and very little about phones but if people wanna work from both of them, I have a lot of catching up with myself to do before I can really put them on par. It's a huge benefit for us that we don't really care what form factor is best for someone cause it's gonna change throughout the day or week because we can support them equally. So I think the hardest parts are either getting the information from where it is right now, whatever legacy systems you're using, into the Zero Trust solution and into the cloud. Or figuring out what are the gaps in our knowledge, right. What do we not know? Oh Alice over there changed departments last week but that's not reflected yet in the system. Well that department change, that job change should reflect the change in authorization access. We don't have that captured, she's gonna have a bad time getting her job done when we move to the system because we actually measure access every time.

>> PEACOCK: Fascinating. So there's definitely cases of companies where, like the old Irish expression, if you're trying to go there I wouldn't have started from here. But for the most part people are able to make this migration without too much trouble.

>> SALTONSTALL: I try to avoid telling people not to go there but it's definitely easier for companies that already have bought in with cloud identity, with a cloud native approach for the applications they're running and with an understanding and appreciation for device management and monitoring.

>> CHUVAKIN: But would you also say it's easy as long as you selected a smallest of apps and a small set of users? So as long as you chose wisely, you initial project may in fact be pretty simple. Like you not gonna choose a mainframe app for phase one. You gonna pick some modern web app internally or in the cloud. So in this case, it would actually be easier right?

>> SALTONSTALL: The funny part is once you've picked the app, once you've gotten to that step, it is actually really easy. It's getting to that step that I see being really hard for a lot of companies that I work with. They get stuck, as Tim said, in some bike shedding. They get stuck in endless security conversations around policy, they get stuck in debates about which kind of authentication and authorization mechanics they wanna use. The hard part is actually reaching that stage where they can flip the switch for that first application. Once they're there, once they've done those precursor steps, it's really easy. It's kind of so easy that they look back and they think, "Wait, we done? That was it? You just click the button?" And it's laughable.

>> CHUVAKIN: So that's the experience in my other sort of coverage areas with security monitory and then log collection, log analysis. I can certainly relate. I hear the same tones from this area to the other one as well. As a kind of interesting sort of transition to this and I don't wanna sound sort of marketing about it but another question I sometimes hear from people in the industry is when they launch the BeyondCorp enterprises product, people started staying, "Oh is this what you launched what Google is doing?" And of course we kinda have to explain that just as chronicle isn't what Google uses internally. This here isn't exactly what Google uses but it's built on the same principles, largely by the same people. So how would you answer it? Is BC the same as Google? No air quotes in podcasts. Sorry.

>> SALTONSTALL: It's okay, we heard them. And I wish it were and I think one day it will be but it's not. What I like to say is that BC was inspired by what we did at Google. What we did is we took many years of development, of mistakes and roadblocks and speed bumps and we turned that into what we think is a much more streamlined journey for our customers. So I do not want any client out there to have to go through a six or seven or eight year process to get to Zero Trust. That is not my goal. And what we did with BeyondCorp Enterprise is turned it into a much simpler and easier to activate system but it also has to be generic. What we built inside was based on the accumulated technical infrastructure and debt that we had over the last 14 or so years and tightly integrated with many of those systems. For example, when we started we had to solve those problems I mentioned earlier. What are the people? What are the devices? Right. We had not solved those problems very well. This is an issue for us. We had four different inventory systems. All doing slightly different measurement of the devices that Google employees used every day. We had to split up our meta inventory system just to corroborate and disentangle the conflicting information. I'm hopeful that most of the people looking for employment at BeyondCorp Enterprise do not have four different inventory services. So there's some good reasons why we didn't just deploy what we built inside. We deployed something that could integrate with a lot of third party tools that companies are already using today could integrate with different identity providers instead of Google's. And it's continuing to grow and evolve based on what we learnt but also based on what we hearing from people who use it.

>> PEACOCK: It's actually how a lot of cloud product management goes is we see what Google has done, we understand the problem that we've solved but we've also solved it over the past n years with this kludge and that patch and these systems held together by some kind of Google duct tape which I think is usually just goops. And when I explain to people, "No, we built this from the ground up for cloud customers to work better." They tend to appreciate it. So on the topic of BCE, what's special about it? Like why is BCE cool? What's going on with Chrome? What's this IAP business I keep hearing about? What's the deal?

>> SALTONSTALL: There's a lot of places where I think it's pretty cool. Identity-aware proxy or IAP is cool because it's part of our global infrastructure and that means although you might think of it as a single point of failure, it's the same one that Google relies on for so many of its user facing services for everything. And that means you're gonna be hitting infrastructure that can handle insane amount of traffic, the biggest denial of service attack attempts the world sees and redundancy across the entire world. So even if there's say an electrical outage across a state or two, you're still gonna be able to get to that proxy that's protecting your important services.

>> PEACOCK: So it's a globally distributed single point of failure?

>> SALTONSTALL: Yeah, sure. That's one way to look at it. Any cloud provider is a globally distributed single point of failure if you look at it in a macro lens, right.

>> PEACOCK: If you look at a very macro, the planet is a global single point of failure.

>> SALTONSTALL: Oh I feel that.

>> PEACOCK: How do we interface with partner endpoint technology? Cause you mentioned cloud sec earlier. Is there nice partner endpoint integration going on?

>> SALTONSTALL: Yeah. So many of the people coming to us looking to adopt this Zero Trust model already have solutions to many of the puzzle pieces. And we don't want them to just throw out the technology integration they've done and the things they've built up already. So we're working with a whole host of alliance partners to figure out how do we pull in the data you're already measuring about those laptops and phones and tablets and watches and the next things. How do we bring that in and feed that into the trust evaluation system so that we can make a smart decision based on the security policy you define without creating more work for you in deploying a brand new agent or in changing the infrastructure you've already built to keep your fleet safe?

>> CHUVAKIN: So let me ask you a possibly uncomfortable question. So we use Chrome as a data source for endpoint inventory and of course we trust Chrome because we built it.

>> SALTONSTALL: And it's open-source.

>> CHUVAKIN: Well there's that. It's easy to verify. So--but you would not say that you trust Chrome more than a partner solution right? Because otherwise it would look strange so we trust the calamity sources on equal ground. Chrome, partner endpoint.

>> SALTONSTALL: I'm gonna actually disagree with the premise of the question. For us, we trust the data from the agents that we built and we deploy on our fleet. For anyone else who wants to run BeyondCorp Enterprise, they're deciding what they trust and don't trust in terms of the data that they're bringing in. The whole point is that you decide what does it require to earn these different gradations of trust and where do you get that data from.

>> CHUVAKIN: So client is perfectly capable of saying, "I trust Tenum but I don't trust Chrome."

>> SALTONSTALL: Yeah sure. You can if you want to.

>> CHUVAKIN: Okay. That's actually good of us.

>> SALTONSTALL: And a lot of the time, if they already built in that we've deployed it to our entire fleet, it's part of our on boarding, it's something that we're monitoring, we're have alert set up, we're locking everything, yeah. You've already got the expertise, never mind the trust, stick with that. It will get you onto Zero Trust faster.

>> PEACOCK: So one more possibly uncomfortable question. Cause what fun would this be without possibly uncomfortable questions for guests? What's the biggest ZT failure you've seen? Has this blown up spectacularly for someone? Is there an endless project that somebody's still working on in basement somewhere but they're missing their stapler? How does it go wrong?

>> SALTONSTALL: I can't think of any newsworthy catastrophes. I think the failure I see is analysis paralysis. And so for example, a company that I can't name but is a technology cloud native, global powerhouse company that we've all heard of cannot seem to get their act together to get this deployed. And they have these technology teams, these really, really sharp technology teams who been debating it for about 9 months and I just wonder what does it take to start down this road? Now I'm exaggerating a little bit because they've done some deployment with some pieces their technology. But in terms of the whole Zero Trust thing and really moving over to a BeyondCorp enterprise or another kind of Zero Trust, they're just stuck. And I'm not quite sure why but to me that seems like a failure cause I know they could do it. I know they have the capability, they have the technology, they have knowledge, they have the skilled engineers. I just don't know what's holding them back.

>> CHUVAKIN: Another analogy from my past life was a security service provider comes to a client and they're discussing monitoring and stuff and the provider says to the client, "Give me list of your critical assets." And the client says, " I wish I had it." Okay fine, so we're on the same page here for sure. But the point is sometimes certain types of context info has to be in place just takes a while to gather. So to me this is not the failure mode as such but it's kind of indeed, it's an analysis paralysis failure mode. You're right.

>> PEACOCK: Well that is it for us on time today. Thank you so much for tuning in and joining once again here on the Cloud Security Podcast. Your guest today was Max Saltonstall. He was a real champ and putting up with all of our relatively kind questioning. Your hosts today were Anton and Tim Peacock. You can find this podcast and it sounds like you already found it on Google Podcasts, Apple Podcasts and wherever else you get your podcasts. You can also follow us on Twitter. Cloudsecpodcast. You can follow Anton, @anton_chuvakin. And myself, @_timpeacock. If you like what you heard today, tweet at us, email us. If you don't like what you heard today, tweet at us or email us. And if we like or don't like what you say, we might even invite you on the show and we promise to at least ask you interesting questions, not necessarily to be nice. Max, thank you once again for joining us.

View more episodes