February 11th, 2018

Spectre, Meltdown, and Virtual Systems

In June of 2017 I wrote a blog for InfoWorld on How to handle the risks of hypervisor hacking. In it, I described the theoretical points where Virtual Machines (VMs) and hypervisors could be hacked. My crystal ball must have been well polished. Spectre and Meltdown prey on one of the points I described there.

What I did not predict is where the vulnerability would come from. As a software engineer, I always think about software vulnerabilities, but I tend to assume that the hardware is seldom at fault. I took one class in computer hardware design thirty years ago. Since then, my working approach is to look first for software flaws and only consider hardware when I am forced, kicking and screaming, to examine for hardware failure. This is usually a good plan for a software engineer. As a rule, when hardware fails, the device bricks (is completely dead), seldom does it continue to function. There is usually not much beyond rewriting drivers that a coder can do to fix a hardware issue. Even rewriting a driver is usually beyond me because it takes more hardware expertise than I have to write a correct driver.

In my previous blog here, I wrote that Spectre and Meltdown probably will not affect individual users much. So far, that is still true, but the real impact of these vulnerabilities is being felt by service providers, businesses, and organizations that make extensive use of virtual systems. Although the performance degradation after temporary fixes have been applied is not as serious as previously estimated, some loads are seeing serious hits and even single digit degradation can be significant is scaled up systems. Already, we’ve seen some botched fixes, which never help anyone.

Hardware flaws are more serious than software flaws for several reasons. A software flaw is usually limited to a single piece of software, often an application. A vulnerability limited to a single application is relatively easy to defend against. Just disable or uninstall the application until it is fixed. Inconvenient, but less of a problem than an operating system vulnerability that may force you to shut down many applications and halt work until the operating system supplier offers a fix to the OS. A flaw in a basic software library can be worse: it may affect many applications and operating systems. The bright side is that software patches can be written and applied quickly and even automatically installed without computer user intervention— sometimes too quickly when the fix is rushed and inadequately tested before deployment— but the interval from discovery of a vulnerability to patch deployment is usually weeks or months, not years.

Hardware chip level flaws cut a wider and longer swathe. A hardware flaw typically affects every application, operating system, and embedded system running on the hardware. In some cases, new microcode can correct hardware flaws, but in the most serious cases, new chips must be installed, and sometimes new sets of chips and new circuit boards are required. If installing microcode will not fix the problem, at the very least, someone has to physically open a case and replace a component. Not a trivial task with more than one or two boxes to fix and a major project in a data center with hundreds or thousands of devices. Often, a fix requires replacing an entire unit, either because that is the only way to fix the problem, or because replacing the entire unit is easier and ultimately cheaper.

Both Intel and AMD have announced hardware fixes to the Spectre and Meltdown vulnerabilities. The replacement chips will probably roll out within the year. The fix may only entail a single chip replacement, but it is a solid prediction that many computers will be replaced. The Spectre and Meltdown vulnerabilities exist in processors deployed ten years ago. Many of the computers using these processors are obsolete, considering that a processor over eighteen months old is often no longer supported by the manufacturer. These machines are probably cheaper to replace than upgrade, even if an upgrade is available. More recent upgradable machines will frequently be replaced anyway because upgrading a machine near the end of its lifecycle is a poor investment. Some sites will put off costly replacements. In other words, the computing industry will struggle with the issues raised by Spectre and Meltdown for years to come.

There is yet another reason vulnerabilities in hardware are worse than software vulnerabilities. The software industry is still coping with the aftermath of a period when computer security was given inadequate attention. At the turn of the 21st century, most developers had no idea that losses due to insecure computing would soon be measured in billions of dollars per year. The industry has changed— software engineers no longer dismiss security as an optional afterthought, but a decade after the problems became unmistakable, we are still learning to build secure software. I discuss this at length in my book, Personal Cybersecurity.

Spectre and Meltdown suggest that the hardware side may not have taken security as seriously as the software side. Now that criminal and state-sponsored hackers are aware that hardware has vulnerabilities, they will begin to look hard to find new flaws in hardware for subverting systems. A whole new world of hacking possibilities awaits.

We know from the software experience that it takes time for engineers to develop and internalize methodologies for creating secure systems. We can hope that hardware engineers will take advantage of software security lessons, but secure processor design methodologies are unlikely to appear overnight, and a backlog of insecure hardware surprises may be waiting for us.

The next year or two promises to be interesting.

January 10th, 2018

Spectre and Meltdown

Will Spectre and Meltdown be the flagship computer security crisis of 2018? There is a good chance that it will be, although I doubt that many personal computer users will be directly affected.

Good news

These flaws are hard to understand and take advanced engineering skills to implement; when implemented they are hard to exploit; I struggle to imagine results that would be worth a hacker’s trouble. Also, exploiting these flaws on a computer you do not already have access to is close to impossible. Consequently, good basic computer hygiene will protect you from these attacks as well as everything else thrown at you. In addition, the exploits are read-only; they do not corrupt data or processes.

The patches are going out this week to all the major operating systems and so far, the bruited predictions of devastating across-the-board 30% performance degradations have not proven out. 10% degradation and only in limited circumstances seems more realistic according to early testing reports.

Less good news

Nevertheless, the fallout from Spectre and Meltdown is likely to cause migraines and insomnia among computer security experts for months, even years to come. And the picture is not quite as rosy for businesses, especially for businesses that rely on virtual computing in various forms, as it is for individuals.


These are not your garden variety zero-day exploits. When I wrote about KRACK a few months ago, I explained that the flaw is particularly bad because it is in the standard and every correct implementation is vulnerable. The Spectre and Meltdown flaws are in the processor chip design. Intel processors have the worst problems and they perform the vast majority of computer processing in the world today, but AMD and ARM processors are also affected. That covers most of the rest of computing, including phones and tablets. For reasons I will elaborate on later, I suspect other processors have not been cited only because no one has looked hard enough yet.

The patches that have been applied are crack sealers; they do not repair the broken foundation that caused the cracks. Fixing the source of the cracks will require new processor designs and new chips. In order to explain just what Spectre and Meltdown are, I have to explain several unfamiliar concepts.

Protection rings

One of the pillars of computer security is called a “protection ring.” They are what prevents one computer process from interfering with another. For example, without protection rings, forcing a user to pass through a login gate before using a computer is easier to circumvent. Protection rings have been built right into the silicon of most processors since the eighties and the concept goes back to the beginnings of multi-processing in the 60s.

To science fiction readers, I liken protection rings to Asimov’s laws of robotics—they are intended to be intrinsic in all computers. In theory, protection rings when properly used make it impossible to break into a well-written operating system without physically altering the processor. When a computer is hacked into, it usually stems from a flaw in the operating system’s use of protection rings, not the physical processor chip.

The Spectre and Meltdown flaws are special because they are gaps in the integrity of privilege rings that were inadvertently built into the processor chips. To see how these gaps were opened, we have to look at concepts of modern processor design.

Multi-core processors

One of these concepts is “multi-core processors.” Before the advent of multi-cores, the capacity of processors was beginning to be limited by the great physical speed limit: the speed of light. When a processor reaches a certain number of instructions per second, it is limited by the time a signal takes to travel across the chip at the speed of light. The processor can’t move on to the next instruction in less time than it takes to read he previous instruction’s results.

Processor designers got around that by putting multiple processors, cores, on a single chip. In theory, by putting two cores on a chip, the speed is doubled. But that does not really solve the problem because taking advantage of the doubled speed requires complex and expensive changes in program design.

Speculative execution

The designers hit on a solution to this: speculative execution. Most computer programs are long chains of “if-thens”. If X condition is met, do Y; if it is not met, do Z. Traditional computers first evaluate X, then decide whether to perform Y or Z. With speculative execution, at the same time one core evaluates X, another core performs Y, and a third performs Z. Depending on how X comes out, Y or Z is discarded. This is a gross simplification, but in the time a single core uses to evaluate X, the three cores already have both the Y and Z results. Thus, the multi-core processor executes a conventionally written program in much less time than a single core. And the speed of computing doubles in 18 months again. Nifty, huh?

Not so nifty. Those discarded speculative chunks of execution can be manipulated in such a way that protection rings are violated. I won’t go into how it’s done. A Google researcher explains it here.

Migraines and insomnia

I am not optimistic when I think about what these defects reveal about processor design. Software development underwent a revolution in the early part of this century when security rose in priority. You can read about it in my book, Personal Cybersecurity. Security was a neglected step-child in the pioneering days of software development in the last century, but around 2000, the industry realized that computing would die if software was not built with more secure methodologies. The revolution is still going on, but the slap-dash attitude toward security that characterized the software cowboys of the 90s is gone.

Spectre and Meltdown tell me that the security revolution did not make it into processor design. Makes you think about why the CEO of Intel sold a big block of Intel stock after the flaws in Intel chips were discovered.

I am afraid we have not heard the last of chip level security flaws. I hope processor designs are not easy pickings for hackers, but the fact that these flaws have been present for at least a decade is daunting. Also, to completely eradicate these flaws, processor chips or entire computers will have to be replaced, which suggests that heads will ache on for years.

Coming soon

I wrote a blog on hypervisor hacking and one on virtual machine security for Network World last year that are affected by the Spectre and Meltdown flaws, but I’ll save comments on the safety of virtual computing for another blog.

December 27th, 2017

Privacy and Online Ads

Without ads monetizing the content of public computer networks, a service that is now low cost would be much more expensive. I’m willing to accept that. But there is something sinister in the online ad business.

Today, “monetize” usually means to change something that is popular in the digital world into a money-maker for someone. Online ads monetize most of what we think of as the internet. Google makes most of their money from online ads as does Facebook. Amazon makes their money from selling things, but their online ads are a crucial part of their business plan.

The ad business has changed

Remember “banner ads”? A seller like Rolex will be glad to pay a premium for a banner ad on a site like the New Yorker that has wide circulation and a good reputation among people with money to spend on luxury watches.

But the banner ad is an endangered species from the age of paper advertising. They are based on high-end, intelligent marketing that made many careers in the 20th Century. But no longer.

21st Century digital advertisers have facts. Traditional marketers knew that New Yorker readers were affluent and well-educated, but they were short on specifics on who was buying and why. Digital marketers today can tell you who sees an ad, how often viewers click on an ad, and, for digital sales, how often they spend money. And they know the age, location, income bracket, and browsing habits of most potential customers. They can target ads to the most likely customers and know exactly how the ads perform.

How do online ads work?

Traditionally, a big city daily newspaper could charge more for their ads than a community weekly because a seller could expect more people to see an ad in the big city daily and act on the ad. Sellers measure the effectiveness of ads by “return on investment” (ROI). If a seller invests $50 in an ad in a community fish wrapper and sees a $100 increase in sales, they get a 200% return. ($100 return/$50 investment = 200%. Sometimes a low-cost ad has better ROI, usually not.

Some businesses occasionally use advertising to improve their image or convey information, but the everyday advertising goal is ROI, using ads to make more sales. The lure of digital advertising is that digital advertising can be fine-tuned to increase ROI by reducing costs and increasing returns.

Digital advertisers can count how many times the ad was seen (impressions) and was followed (clicks). If the transaction is digital, they can count the number of times the ad resulted in a sale. Traditional paper advertising only knows how many copies of the ad were circulated, not how often the ad was seen and only generalities about readers.

The network collects information on buyers that can be used to target advertising toward people likely to buy. For example, people who don’t have cars are unlikely to buy car polish. Therefore, car polish sellers can improve their advertising ROI by directing their ads to car owners and ignoring people without cars.

Who are the players in the online ad biz?

  • Customers. That’s you.
  • The ad publishers. Google, Facebook, Amazon, etc. Ad publishers put the ads in front of potential customers.
  • Ad networks and exchanges. The folks in the background who match likely buyers to sellers and maximize the vigorish. When you open a web page with slots for ad, the slots are often auctioned off highest bidder in milliseconds. The bidders use information about you, to decide how much to bid. You may be familiar with some of these players like “DoubleClick” whose addresses flash by as you enter a site.
  • Ad agencies. Those waggish artists who think up cunning ads for the advertisers. These companies usually have bland names like “WPP Group.”
  • Data brokers. The vacuum cleaners that suck up data and sort it into a commodity they can sell to advertisers, ad agencies, networks, and exchanges. These are companies like Blue Kai or Live Ramp, whom you may not have heard of.

Except for customers, the players are often combined. There are one-stop shops that combine all the functions and boutiques that specialize in a narrow aspect of the process.

The network never forgets

The data collected on buying habits has grown rapidly in the last few years. If you do something on the network, someone, somewhere, has taken a note. The more we use computer networks, the more data is amassed on us. “Big data” arose to process the mountains of accumulated data.

Today, electronic payment is common, and many customers get discounts by identifying themselves when they purchase. Consequently, grocery store managers may know more about your food buying habits than you do. They can use that information to offer the items you want, but they also use it to find and persuade you to buy more profitable items. They can appeal to habits you may not even know you have. Online sales are even more effective at collecting data on customers.

Although you may not enjoy being manipulated in this way, most people still choose to use payment methods that identify themselves and trade their phone number at the point-of-sale for reduced prices. A lot of people feel that the convenience of electronic payment and a reduced price are reasonable tradeoff for subjecting themselves to manipulation by their sellers.

Why do online ads make me feel uneasy?

Using network habits to target ads is occasionally annoying. My grandfather died of colon cancer after a colostomy fifty years ago. Recently I wondered how those ugly colostomy bags had changed. I searched online. What a mistake! I still occasionally get an ad for disposable bags in cheery prints.

Creepy, yes, but not threatening. I, thank Heavens, am not remotely likely to purchase a colostomy bag according to my gastroenterologist. The sellers have made a mistake, but it only costs them a few cents and they certainly get a worthwhile ROI on their ads, winning the numbers game. And I get annoying ads. Nothing to lose sleep over.

Misuse of personal profiles

But let’s change the story some. Suppose you looked up alcoholism treatment out of curiosity. And the user of your profile was not an alcoholism treatment center selling their services, but an investigative agency running a check for a potential employer to whom you sent an application. Maybe the job was important to you and you were well-qualified, but your application was tossed on the first round because you were flagged as an alcoholic.

Do you see how the situation changed? A seller looking at ROI doesn’t grudge a fried fig for a few ads sent to the wrong place. A loss of a few cents to misdirected ads is nothing compared to all those colostomy bag sales. But you lost a job that you may have wanted, even needed, badly. And the potential employer lost a brilliant prospect. This happens when a personal profile is used in a scenario where much harm can result from inferences that are perfectly valid in other circumstances.

The danger is that the profiles will applied wrongly when they are perfectly valid and useful in most circumstances. That is sinister.

November 30th, 2017

Cyber Defense Skill: URL Reading

Want to quickly sort out real emails from spam? Spot a bad links on web pages? Identify sham web sites? I have a suggestion: learn to read URLs.

Learning to read URLs is like taking a class in street self-defense or carrying a can of mace. Actually, much better because reading URLs can’t be turned against you. You might end up in the hospital or worse if you resist a street thug with your self-defense skills, but you will never be injured spotting a bad URL.

Uniform Resource Locators (URLs), more properly called Uniform Resource Identifiers (URIs), direct all the traffic on the World Wide Web. Almost every cyber-attack directs traffic to or from an illegitimate URL at some point in the assault. If you can distinguish a good address from a bad address and develop the habit of examining internet addresses, you will be orders of magnitude more difficult to hack.

Addresses are constructed according to simple rules. You can master the rules you need to know in order to distinguish legitimate addresses from scams in a few minutes. And be much safer.

If you want to dig deep into URLs, take a look at RFC 3986. There is much more to URLs than I cover here.

Here is a typical simple URL:



The first part, called the scheme, “http:” tells you that it is a HyperText Transfer Protocol (HTTP) address. You need to know two things about the HTTP scheme. First, almost all data on the web travels to and from your desktop, laptop, tablet, or phone over HTTP. In fact, if an address does not begin with “http”, it’s not a web address. There other schemes, the most important of these is “mailto:”, which designates an email address. More on this below.

Secure HTTP

There is an important variant of HTTP called HTTPS. The “S” stands for “secure.” Data shipped via HTTPS is encrypted and the source and destination are verified with a security organization. HTTPS used to be reserved for financial transactions, but now, with all the dangers of the network, HTTPS is encouraged for all traffic. When you see “https” in a web address, hackers have a hard time snooping on your data or faking a web site. HTTPS is especially important if you are on open public WiFi at a coffee shop or other public place.

Not too long ago, security experts said HTTPS guaranteed that a site was legitimate. That is no longer good advice. HTTPS is not a guarantee that a site is legit. Smart scamming hackers can set up fake sites with HTTPS security. You have to check the rest of the address for signs of bogosity. However, setting up a fake site with a legitimate address is still hard, so a good address with HTTPS is still a strong bet.

HTTP address “authority”

The part of the address following the “//” is the “authority.” Most of the time, the authority is a registered domain name. The authority section of a URL ends with a “/”. Notice that the slash leans forward, not backward. A backward slash is completely different. The “query” follows the forward slash. The query usually contains search criteria that narrow down the data you want retrieved and is often hard to interpret without specific information about the domain. You can ignore it, although sometimes hackers can learn secrets about a web site from information inadvertently placed in the query.

Domain extensions

In the above address, “marvinwaschke.com” is a domain name that I have registered with the with the Internet Assigned Number Authority (IANA). “.com” is the extension. In the old days, there were only a few extensions allowed: “.gov”, “.edu”, “.net”, “.com”, and “.mil”. They are still the most common, although many others— such as “.tv”, “.partners”, “.rocks” and country abbreviations— have been added.

You can use extensions as a clue. For instance, most established firms and organizations still use the old standbys. A web site with a “amex.rocks” domain is likely not the American Express you think it is. We all know that some countries harbor more hackers than others. If an address has an extension that is an abbreviation for a cyber rogue state, be careful.

Remember, these are clues, not rules. A street lined with wrecked cars and broken windows may be crime free, but more often than not, it is a dangerous neighborhood. The same applies to incongruous domain names. They could be safe, but there is a good chance they are not.

Authority subsections

The authority section is divided by periods (“.”s) and reads in reverse. The extension that immediately precedes the first forward slash is the most important. “.com” in “marvinwascke.com” indicates that the marvinwaschke.com domain is in the vast segment of the internet made up of commercial ventures. “marvinwaschke” determines which commercial venture the address refers to. “www” indicates that the address points to the “www” part of the “marvinwaschke” venture. I could set up my website to have a “public.marvinwaschke.com” section or a “public.security.marvinwascke.com” section if I cared to. The “www” is historically so common, most browsers will strip it off or add it on as needed to make a connection.

“Microsoft.marvinwaschke.com” only indicates that my web site has a section devoted to Microsoft. “Microsoft.marvinwaschke.com” has nothing to do with Microsoft Corporation. Hackers make use of this to try to fool you that “Microsoft.pirates-r-us.ru” is a Microsoft site. It’s not! Hackers are creative. Make sure that the right end of the domain name makes sense.

Email URIs

Email addresses are URIs that follow a different scheme but use the same domain name rules. Usually, email addresses drop the “mailto” scheme but they can always be fully written out like mailto://boss@example.com. If you see an address like captain@microsoft.pirates-r-us.ru you can be fairly certain that the mail did not come from Bill Gates.

When in doubt, Google it

When you see a link or address with a suspicious domain name, Google the domain name before you use the address. Most of the time, Google will pick up information on dangerous domains.

Look at every link with caution

The internet is all about grabbing your attention. Absurd promises abound that that few people would take seriously after they took a moment to think. Losing weight is hard, wealth management is useless if you aren’t already accumulating wealth the hard way, and no miracle food will prevent cancer or make you a genius. Not all ads are scams, but  don’t tempt fate by clicking on links that prey on impossible hopes.


Make a habit of looking at internet addresses. If you place the cursor over a link or address, most browsers and email tools will display the working address. Look at the address. Does something look wrong? If so, use care. The habit of looking at addresses will make you much harder to hack than unsavvy computer users.