Yahoo has had a rough year.
Last September, the company announced a state-sponsored data breach that affected more than 500 million accounts, and followed that up a few months later by disclosing a separate hack of more than one billion accounts. To add financial insult to injury, the company's $4.8 billion acquisition offer from Verizon was hit with a $350 million discount in the wake of the breach disclosures amidst reports of Verizon trying to wiggle out of the deal.
Then to top it all off, Verizon pulled a Tronc and renamed the newly merged AOL and Yahoo brands as Oath.
In the intervening months, we've learned more about the breaches. They occurred way back in 2013 and 2014, respectively, and—in what's becoming a high-profile theme for US data breaches—it turns out Russia was behind it. Two Russian hackers have now been indicted for one of the breaches. But Yahoo's security woes aren't over. In March, the company disclosed yet another 32 million breached accounts, and began notifying users about "forged cookie" attacks.
It's safe to say that since coming on as Yahoo's Chief Information Security Officer (CISO) in 2015, Bob Lord has been busy. At TechCrunch Disrupt today in New York, Lord walked through Yahoo's nightmare year and talked about how they traced the breaches back to Russia, the new endpoint security and other countermeasures the company has put in place, and how the controversy affected the Verizon deal.
How Yahoo Discovered the Breach
"We have the benefit of a group within our organization that specializes in tracking down [Advanced Persistent Threat] APT attacks. So we had world-class people who knew what to look for and how to chase down leads to figure out who was behind these attacks," said Lord. "We're required to regularly look for information—things that are traded in various places on the web. You see tweets now and then, and often it's a double of something or recycled data from another dump, but this was very different. We saw it in a data dump and jumped right in.
What It Was Like at Yahoo After the Breach
"At the end of last year, we sent out a billion and a half emails," said Lord. "It was like that effect Alfred Hitchcock perfected where things are telescoping out but you can see everything. I remember feeling that when I was putting all the pieces together. It wasn't a great feeling."
What Took Yahoo So Long to Figure it Out?
A number of weeks passed last fall when Yahoo was working to understand the extent of the hack and determine what happened. The company brought in outside forensic experts and traced on of the hacks back to 2014.
"These campaigns can run for an extended period of time," said Lord. "These aren't smash-and-grab attacks. They're long-term plays. We commissioned a study to go back in time and put all the pieces together. It's all in our most recent 10-K filing. There is a section that goes into all the major elements of the breach and what happened internally with the evidence we found.
"They worked hard to fly under the radar and get the access they were specifically tasked with," Lord went on. "It's now clear in hindsight that these guys could have gotten actual jobs. They were very good. Modifying production systems is hard when you're trained and have supervision. It's a difficult thing to pull off without detection. I stay away from the word sophisticated, but these were skilled individuals going back and forth between criminal and state-sponsored activities."
How Russia Was Involved
"The indictment is worth your time. [The Department of] Justice is alleging that these attackers conducted a series of operations that included attacks against Yahoo and its infrastructure. These are FSB [what used to the the KGB] intelligence officers working within the government that tasked two hackers, one in Canada awaiting trial and one in Russia," said Lord.
"This is a remarkable story," Lord went on. "We have visibility up to a certain point about the lengths they went to to attack the infrastructure and get information about our users. That's unprecedented. I'm unaware of any other case where Russia has indicted FSB officers. The criminals also engaged in a series of activities for their own financial gain. You really can't make this stuff up."
How the Hackers Got All That Data
"There's a specific set of steps attackers have to go through in order to achieve their goals," said Lord. "They had to do initial reconnaissance to see what type of servers are out there, look for footholds, and perform an initial intrusion. Then from there they have to elevate their privileges and move laterally. Whatever machines they break into are 99 percent of the time not the ones they want, so they have to move from machine to machine flying under the radar to get what they're looking for."
What Yahoo Is Doing About It
"Part of the reason I took this job is because we had the APT group already set up. There was that and a red team—which uses all the same tactics and tools of real attackers and attempts to penetrate our infrastructure—and the red team always wins," said Lord. "It's important for us to understand how attacks happen so we can build up fortifications. Companies engage in routine best practices, but it's like practicing martial arts in a mirror instead of in the ring. It's hard to prove a negative, but what you can do is build up a preponderance of circumstantial evidence. All the things we're doing internally to provide signals that these attacks and exploits would not be possible today."
How Former CEO Marissa Mayer Handled the Crisis
"Culture starts at the top, but it's a dynamic and living thing. One person isn't responsible," said Lord. "My experience was of the CEO being at the forefront of the investigation. We jumped right in. When I look at the hiring we've been able to do, we got the support we needed."
How the Breaches Affected the Verizon Deal
"Security professionals are rarely surprised when this kind of thing happens. If you've been in the business for more than a few years, you've had your skirmishes. It's about whether you can get enough of a root cause analysis to demonstrate there are improvements in place and the attackers are not in the network. Those are the basic questions we had to answer," said Lord.
"I was very impressed with Verizon and AOL's leadership, and how they were focused on this but also knowledgeable enough not to make it an emotional decision. We have common enemies; all the major companies have major adversaries. So that's how that went."