Spectre, Meltdown, and Virtual Systems

In June of 2017 I wrote a blog for InfoWorld on How to handle the risks of hypervisor hacking. In it, I described the theoretical points where Virtual Machines (VMs) and hypervisors could be hacked. My crystal ball must have been well polished. Spectre and Meltdown prey on one of the points I described there.

What I did not predict is where the vulnerability would come from. As a software engineer, I always think about software vulnerabilities, but I tend to assume that the hardware is seldom at fault. I took one class in computer hardware design thirty years ago. Since then, my working approach is to look first for software flaws and only consider hardware when I am forced, kicking and screaming, to examine for hardware failure. This is usually a good plan for a software engineer. As a rule, when hardware fails, the device bricks (is completely dead), seldom does it continue to function. There is usually not much beyond rewriting drivers that a coder can do to fix a hardware issue. Even rewriting a driver is usually beyond me because it takes more hardware expertise than I have to write a correct driver.

In my previous blog here, I wrote that Spectre and Meltdown probably will not affect individual users much. So far, that is still true, but the real impact of these vulnerabilities is being felt by service providers, businesses, and organizations that make extensive use of virtual systems. Although the performance degradation after temporary fixes have been applied is not as serious as previously estimated, some loads are seeing serious hits and even single digit degradation can be significant is scaled up systems. Already, we’ve seen some botched fixes, which never help anyone.

Hardware flaws are more serious than software flaws for several reasons. A software flaw is usually limited to a single piece of software, often an application. A vulnerability limited to a single application is relatively easy to defend against. Just disable or uninstall the application until it is fixed. Inconvenient, but less of a problem than an operating system vulnerability that may force you to shut down many applications and halt work until the operating system supplier offers a fix to the OS. A flaw in a basic software library can be worse: it may affect many applications and operating systems. The bright side is that software patches can be written and applied quickly and even automatically installed without computer user intervention— sometimes too quickly when the fix is rushed and inadequately tested before deployment— but the interval from discovery of a vulnerability to patch deployment is usually weeks or months, not years.

Hardware chip level flaws cut a wider and longer swathe. A hardware flaw typically affects every application, operating system, and embedded system running on the hardware. In some cases, new microcode can correct hardware flaws, but in the most serious cases, new chips must be installed, and sometimes new sets of chips and new circuit boards are required. If installing microcode will not fix the problem, at the very least, someone has to physically open a case and replace a component. Not a trivial task with more than one or two boxes to fix and a major project in a data center with hundreds or thousands of devices. Often, a fix requires replacing an entire unit, either because that is the only way to fix the problem, or because replacing the entire unit is easier and ultimately cheaper.

Both Intel and AMD have announced hardware fixes to the Spectre and Meltdown vulnerabilities. The replacement chips will probably roll out within the year. The fix may only entail a single chip replacement, but it is a solid prediction that many computers will be replaced. The Spectre and Meltdown vulnerabilities exist in processors deployed ten years ago. Many of the computers using these processors are obsolete, considering that a processor over eighteen months old is often no longer supported by the manufacturer. These machines are probably cheaper to replace than upgrade, even if an upgrade is available. More recent upgradable machines will frequently be replaced anyway because upgrading a machine near the end of its lifecycle is a poor investment. Some sites will put off costly replacements. In other words, the computing industry will struggle with the issues raised by Spectre and Meltdown for years to come.

There is yet another reason vulnerabilities in hardware are worse than software vulnerabilities. The software industry is still coping with the aftermath of a period when computer security was given inadequate attention. At the turn of the 21st century, most developers had no idea that losses due to insecure computing would soon be measured in billions of dollars per year. The industry has changed— software engineers no longer dismiss security as an optional afterthought, but a decade after the problems became unmistakable, we are still learning to build secure software. I discuss this at length in my book, Personal Cybersecurity.

Spectre and Meltdown suggest that the hardware side may not have taken security as seriously as the software side. Now that criminal and state-sponsored hackers are aware that hardware has vulnerabilities, they will begin to look hard to find new flaws in hardware for subverting systems. A whole new world of hacking possibilities awaits.

We know from the software experience that it takes time for engineers to develop and internalize methodologies for creating secure systems. We can hope that hardware engineers will take advantage of software security lessons, but secure processor design methodologies are unlikely to appear overnight, and a backlog of insecure hardware surprises may be waiting for us.

The next year or two promises to be interesting.

Spectre and Meltdown

Will Spectre and Meltdown be the flagship computer security crisis of 2018? There is a good chance that it will be, although I doubt that many personal computer users will be directly affected.

Good news

These flaws are hard to understand and take advanced engineering skills to implement; when implemented they are hard to exploit; I struggle to imagine results that would be worth a hacker’s trouble. Also, exploiting these flaws on a computer you do not already have access to is close to impossible. Consequently, good basic computer hygiene will protect you from these attacks as well as everything else thrown at you. In addition, the exploits are read-only; they do not corrupt data or processes.

The patches are going out this week to all the major operating systems and so far, the bruited predictions of devastating across-the-board 30% performance degradations have not proven out. 10% degradation and only in limited circumstances seems more realistic according to early testing reports.

Less good news

Nevertheless, the fallout from Spectre and Meltdown is likely to cause migraines and insomnia among computer security experts for months, even years to come. And the picture is not quite as rosy for businesses, especially for businesses that rely on virtual computing in various forms, as it is for individuals.

Scope

These are not your garden variety zero-day exploits. When I wrote about KRACK a few months ago, I explained that the flaw is particularly bad because it is in the standard and every correct implementation is vulnerable. The Spectre and Meltdown flaws are in the processor chip design. Intel processors have the worst problems and they perform the vast majority of computer processing in the world today, but AMD and ARM processors are also affected. That covers most of the rest of computing, including phones and tablets. For reasons I will elaborate on later, I suspect other processors have not been cited only because no one has looked hard enough yet.

The patches that have been applied are crack sealers; they do not repair the broken foundation that caused the cracks. Fixing the source of the cracks will require new processor designs and new chips. In order to explain just what Spectre and Meltdown are, I have to explain several unfamiliar concepts.

Protection rings

One of the pillars of computer security is called a “protection ring.” They are what prevents one computer process from interfering with another. For example, without protection rings, forcing a user to pass through a login gate before using a computer is easier to circumvent. Protection rings have been built right into the silicon of most processors since the eighties and the concept goes back to the beginnings of multi-processing in the 60s.

To science fiction readers, I liken protection rings to Asimov’s laws of robotics—they are intended to be intrinsic in all computers. In theory, protection rings when properly used make it impossible to break into a well-written operating system without physically altering the processor. When a computer is hacked into, it usually stems from a flaw in the operating system’s use of protection rings, not the physical processor chip.

The Spectre and Meltdown flaws are special because they are gaps in the integrity of privilege rings that were inadvertently built into the processor chips. To see how these gaps were opened, we have to look at concepts of modern processor design.

Multi-core processors

One of these concepts is “multi-core processors.” Before the advent of multi-cores, the capacity of processors was beginning to be limited by the great physical speed limit: the speed of light. When a processor reaches a certain number of instructions per second, it is limited by the time a signal takes to travel across the chip at the speed of light. The processor can’t move on to the next instruction in less time than it takes to read he previous instruction’s results.

Processor designers got around that by putting multiple processors, cores, on a single chip. In theory, by putting two cores on a chip, the speed is doubled. But that does not really solve the problem because taking advantage of the doubled speed requires complex and expensive changes in program design.

Speculative execution

The designers hit on a solution to this: speculative execution. Most computer programs are long chains of “if-thens”. If X condition is met, do Y; if it is not met, do Z. Traditional computers first evaluate X, then decide whether to perform Y or Z. With speculative execution, at the same time one core evaluates X, another core performs Y, and a third performs Z. Depending on how X comes out, Y or Z is discarded. This is a gross simplification, but in the time a single core uses to evaluate X, the three cores already have both the Y and Z results. Thus, the multi-core processor executes a conventionally written program in much less time than a single core. And the speed of computing doubles in 18 months again. Nifty, huh?

Not so nifty. Those discarded speculative chunks of execution can be manipulated in such a way that protection rings are violated. I won’t go into how it’s done. A Google researcher explains it here.

Migraines and insomnia

I am not optimistic when I think about what these defects reveal about processor design. Software development underwent a revolution in the early part of this century when security rose in priority. You can read about it in my book, Personal Cybersecurity. Security was a neglected step-child in the pioneering days of software development in the last century, but around 2000, the industry realized that computing would die if software was not built with more secure methodologies. The revolution is still going on, but the slap-dash attitude toward security that characterized the software cowboys of the 90s is gone.

Spectre and Meltdown tell me that the security revolution did not make it into processor design. Makes you think about why the CEO of Intel sold a big block of Intel stock after the flaws in Intel chips were discovered.

I am afraid we have not heard the last of chip level security flaws. I hope processor designs are not easy pickings for hackers, but the fact that these flaws have been present for at least a decade is daunting. Also, to completely eradicate these flaws, processor chips or entire computers will have to be replaced, which suggests that heads will ache on for years.

Coming soon

I wrote a blog on hypervisor hacking and one on virtual machine security for Network World last year that are affected by the Spectre and Meltdown flaws, but I’ll save comments on the safety of virtual computing for another blog.

Cyber Defense Skill: URL Reading

Want to quickly sort out real emails from spam? Spot a bad links on web pages? Identify sham web sites? I have a suggestion: learn to read URLs.

Learning to read URLs is like taking a class in street self-defense or carrying a can of mace. Actually, much better because reading URLs can’t be turned against you. You might end up in the hospital or worse if you resist a street thug with your self-defense skills, but you will never be injured spotting a bad URL.

Uniform Resource Locators (URLs), more properly called Uniform Resource Identifiers (URIs), direct all the traffic on the World Wide Web. Almost every cyber-attack directs traffic to or from an illegitimate URL at some point in the assault. If you can distinguish a good address from a bad address and develop the habit of examining internet addresses, you will be orders of magnitude more difficult to hack.

Addresses are constructed according to simple rules. You can master the rules you need to know in order to distinguish legitimate addresses from scams in a few minutes. And be much safer.

If you want to dig deep into URLs, take a look at RFC 3986. There is much more to URLs than I cover here.

Here is a typical simple URL:

https://www.marvinwaschke.com

HTTP

The first part, called the scheme, “http:” tells you that it is a HyperText Transfer Protocol (HTTP) address. You need to know two things about the HTTP scheme. First, almost all data on the web travels to and from your desktop, laptop, tablet, or phone over HTTP. In fact, if an address does not begin with “http”, it’s not a web address. There other schemes, the most important of these is “mailto:”, which designates an email address. More on this below.

Secure HTTP

There is an important variant of HTTP called HTTPS. The “S” stands for “secure.” Data shipped via HTTPS is encrypted and the source and destination are verified with a security organization. HTTPS used to be reserved for financial transactions, but now, with all the dangers of the network, HTTPS is encouraged for all traffic. When you see “https” in a web address, hackers have a hard time snooping on your data or faking a web site. HTTPS is especially important if you are on open public WiFi at a coffee shop or other public place.

Not too long ago, security experts used to say HTTPS guaranteed that a site was legitimate. That is no longer good advice. HTTPS is not a guarantee that a site is legit. Smart scamming hackers can set up fake sites with HTTPS security. You have to check the rest of the address for signs of bogosity. However, setting up a fake site with a legitimate address is still hard, so a good address with HTTPS is still a strong bet.

HTTP address “authority”

The part of the address following the “//” is the “authority.” Most of the time, the authority is a registered domain name. The authority section of a URL ends with a “/”. Notice that the slash leans forward, not backward. A backward slash is completely different. The “query” follows the forward slash. The query usually contains search criteria that narrow down the data you want retrieved and is often hard to interpret without specific information about the domain. You can ignore it, although sometimes hackers can learn secrets about a web site from information inadvertently placed in the query.

Domain extensions

In the above address, “marvinwaschke.com” is a domain name that I have registered with the with the Internet Assigned Number Authority (IANA). “.com” is the extension. In the old days, there were only a few extensions allowed: “.gov”, “.edu”, “.net”, “.com”, and “.mil”. They are still the most common, although many others— such as “.tv”, “.partners”, “.rocks” and country abbreviations— have been added.

You can use extensions as a clue. For instance, most established firms and organizations still use the old standbys. A web site with a “amex.rocks” domain is likely not the American Express you think it is. We all know that some countries harbor more hackers than others. If an address has an extension that is an abbreviation for a cyber rogue state, be careful.

Remember, these are clues, not rules. A street lined with wrecked cars and broken windows may be crime free, but more often than not, it is a dangerous neighborhood. The same applies to incongruous domain names. They could be safe, but there is a good chance they are not.

Authority subsections

The authority section is divided by periods (“.”s) and reads in reverse. The extension that immediately precedes the first forward slash is the most important. “.com” in “marvinwascke.com” indicates that the marvinwaschke.com domain is in the vast segment of the internet made up of commercial ventures. “marvinwaschke” determines which commercial venture the address refers to. “www” indicates that the address points to the “www” part of the “marvinwaschke” venture. I could set up my website to have a “public.marvinwaschke.com” section or a “public.security.marvinwascke.com” section if I cared to. The “www” is historically so common, most browsers will strip it off or add it on as needed to make a connection.

“Microsoft.marvinwaschke.com” only indicates that my web site has a section devoted to Microsoft. “Microsoft.marvinwaschke.com” has nothing to do with Microsoft Corporation. Hackers make use of this to try to fool you that “Microsoft.pirates-r-us.ru” is a Microsoft site. It’s not! Hackers are creative. Make sure that the right end of the domain name makes sense.

Email URIs

Email addresses are URIs that follow a different scheme but use the same domain name rules. Usually, email addresses drop the “mailto” scheme but they can always be fully written out like mailto://boss@example.com. If you see an address like captain@microsoft.pirates-r-us.ru you can be fairly certain that the mail did not come from Bill Gates.

Near miss URIs

A favorite hacking trick is to register a domain that looks real, but is just a little off. For example, micrasoft.com instead of microsoft.com. Keep an eye out for those little tricks.

When in doubt, Google it

When you see a link or address with a suspicious domain name, Google the domain name before you use the address. Most of the time, Google will pick up information on dangerous domains.

Look at every link with caution

The internet is all about grabbing your attention. Absurd promises abound that that few people would take seriously after they took a moment to think. Losing weight is hard, wealth management is useless if you aren’t already accumulating wealth the hard way, and no miracle food will prevent cancer or make you a genius. Not all ads are scams, but  don’t tempt fate by clicking on links that prey on impossible hopes.

Finally

Make a habit of looking at internet addresses. Often, a link on a webpage or in an email is text like ” here “.  Hackers hide bogus URLs under innocuous text. They also sometimes use a legitimate URL for the text and stick in a dubious URL for the real target.  Like this: https://marvinwaschke.com  If you place the cursor over a link or address, most browsers and email tools will display the working address in the lower left-hand corner of the window. Look at the address remembering all the cautions in this post. Does something look wrong? If so, use care. Try the two links in this paragraph to see what I mean. The habit of looking at addresses will make you much harder to hack than unsavvy computer users.

Ransomware Protection Strategies for Small Business

I was chatting with a lawyer yesterday about cybersecurity and he mentioned that he has heard that law offices in our county have been hit with ransomware in the last few months. Law offices are a ripe target for ransomware because the confidentiality and integrity of their records are vital. Lose their records, lose their business. The same applies to many other small businesses.

What is ransomware? Ransomware is malicious software used by a criminal to deny the rightful owner of a computer system access to vital system resources and demand payment to restore the resources. Usually, ransomware encrypts data and demands Bitcoin or other untraceable cybercurrency payment for decrypting the data.

What should these offices and other small business do to protect themselves from ransomware? I suggest a two-pronged approach: prevention and damage control.

Prevention

Take steps to avoid a ransomware assault in the first place. The practices below are basic cyber hygiene for everyone that will lessen the chances of all forms of cybercrime.

  • Use a good anti-virus scanning utility. Keep it up-to-date and scan regularly.

    Wondering which utility? Windows Defender, the default Windows 10 anti-virus is a good choice. It’s already installed, doesn’t get in the way, and does a competent job. Are 3rd party tools better? The anti-virus business is highly competitive. Which utility is best changes rapidly. I use Windows Defender myself because it is convenient, and Microsoft has invested in keeping Defender among the best, which is good enough for me. Whatever you do, use an anti-virus utility and keep it up to date.

  • Use only supported operating systems and applications and subscribe to automatic updates. New vulnerabilities show up every day. Accept the manufacturer’s help in patching up the holes as the appear.

    If you don’t trust your vendor’s updates, get rid of their software. If you don’t, you put your business at risk. The only exception to this rule is when you have special software that is frequently broken by security patches. At that point, you are strapped and dependent on the maintainer of your special software. Avoid this situation if you can.

  • Be cautious of links in web pages, emails, and messages. If a link looks dodgy, skip it. Be doubly cautious about attachments to emails and messages. If you are not sure where something came from, don’t open it. If there is a question, call the sender and confirm that it is legit. Links and attachments are the most common entry points for ransomware.

Damage control

If you are diligent in following these three practices, a criminal will have a hard time entering your computer system and might pass it by for easier prey, but you have no guarantee. Let your guard down an instant and you are vulnerable. A smart criminal who is intent on assaulting your system is likely to eventually succeed no matter what you do. However, if you plan ahead, the game is not over when you get a ransom note. Your backups are critical in recovering from a ransomware assault and a lot of other computer system mishaps.

  • Backup your system regularly. I favor reputable cloud backup services because they tend to be automated and trouble free. The most likely time for ransomware to hit is the day someone forgot to run backups, or the janitor switched off the external backup drive by mistake.
  • Test your backup system regularly. All backup systems are complex mechanisms that sometimes fail. Your only assurance that they are working is a recent successful test. I always assume that a backup system that has not been tested recently does not work. I have seen disasters in the aftermath of backup systems that were assumed to be working but were not.
  • Protect your backups. Smart ransomware attempts to mash your backups. Put up barriers to protect them. Check the documentation on your system or talk to your IT technician on how to do it effectively.
  • Have a plan. A rock-solid backup system is the foundation for recovery but consider what you will do the instant a ransom note pops up. I suggest immediately ceasing all activity, detaching from all external networks, and running a virus scan. Then contact an experienced technician for help. Do not shut the system down or restart if you can avoid it. Some recovery methods depend on recovering data from memory that disappears on shutdown or reboot.

Call law enforcement

Local law enforcement may not be able to help because the criminal is likely to be in a different state or country. Keep them informed anyway. Unreported crimes encourage law breakers. Some states have cyber crime task forces with real muscles that work with the FBI and the Department of Homeland Security to shut these operations down. If local law enforcement can’t help, report the crime to the FBI’s Internet Crime Complaint Center. (IC3) If cyber crimes are not reported, funds will not be allocated to fight cyber crime and laws will not be written or changed to reflect the injuries done by these criminals.

Consider cyber insurance

Cybercrime is not that different from conventional theft and damage. I understand that cyber business insurance is becoming more common. I am not familiar with the costs involved or the efficacy of the policies, but your business insurance agent is likely to be able to help. Nonetheless, remember that avoiding or controlling damage is less disruptive to business than insurance compensation and insurance seldom makes up the whole cost of an assault.

A final note

Ransomware and other forms of cyber crime are real threats. In 2016, over 1.3 billion dollars in losses were reported to the FBI. Those who take steps to protect their business will suffer less and may completely avoid becoming victims.