My Blood Ran Cold This Morning

I live 4 miles from one oil refinery and 6 miles from another. I don’t think about it much. But I did this morning when I read an article in the MIT Technology Review on the Triton malware.

I don’t know much about the two refineries. Forty years ago, when I was a carpenter I worked for a few months on a construction project inside one of them. That was probably the safest construction site I ever worked on. There, we followed safety rules unheard of on other sites. Forget to snap on your safety rope doing high work and somebody would yell at you before you got your hammer out of your belt.

And the rules had teeth. As I remember, intentionally break a safety rule and you were outside the gate, which was not trivial. Industrial carpenter work paid way more than residential or commercial work and double-time overtime was regular. I think I got triple-time for working overtime on a Fourth of July.

That was in the early 80s when OSHA work rules were a cat and mouse game construction workers played for fun. If I had followed OSHA rules regularly then, instead of trying to avoid them, or landed more refinery jobs where they were serious about the rules, I probably wouldn’t wear a hearing device today and fewer of my brother carpenters would be missing body parts or have died from asbestos cancer.

So. I have not thought much about safety in the refineries. I’ve seen my share of sloppy computer security in industrial plants of in past years: default passwords, work stations left unlocked without attendants, and unpatched or outdated software were all common in plants where products I worked on were installed, but I never connected the dots to the refineries next door, which I thought of as paragons of safety.

I did this morning.

Triton is malware that was discovered in a refinery in Saudi Arabia. Hackers breached plant computer security in 2014 and began to infiltrate the system with the Triton malware. They caused a plant shutdown in June 2017, which raised suspicions but did not lead to detection. A second shutdown in August 2017 revealed the Triton attack. Neither of the shutdowns caused more than inconvenience, but the subsequent investigation revealed that the hackers were prepared to wreak massive damages and deaths.

To explain how Triton works, I have to explain how modern industrial control works today. In some ways, industrial plants are much safer than they were when I was a carpenter. Something called a SCADA (Supervisory Control And Data Acquisition) controls many industrial processes today.

One summer during college, I was SCADA in a pea freezing plant. They called me a “tunnel man.” My job was to walk a circuit in a refrigerated freezing belt tunnel, checking temperatures, salinity levels, and progress over the belts. I reported to the refrigeration engineers several times an hour with the measurements. If something jammed the belt, I had to sound the alarm so that the engineers could turn down the refrigeration units so the belt would not freeze up solid.

In those days, I was fairly responsible, and I took my job seriously. I realized that if I messed up, the damage could be great. However, the truth was that the job was totally mindless and boring after the first week. A trained pigeon could have done the job at least as well as I did.

The old pea freezing plant closed decades ago, but today the tunnel man would be replaced by SCADA. Sensors would relay temperatures and salinity to a control dashboard, other devices would detect issues on the belt, and most of the control would be automated. A SCADA system monitors continuously instead of my periodic inspections and react quicker and more precisely than the engineers listening to some kid inaccurately describe ice buildup on the Kelly belt. Freezing plants may still have tunnel men as backups, although with labor costs today, I doubt it.

SCADA systems are not perfect, but they are much better than humans or trained pigeons for mindless relay of information and rote response. Each year SCADA, with the help of advancing sensing and control algorithms, gets better. But suppose, someone tampers with the sensors? Or the control rules? Even in an innocuous vegetable freezing plant, an exploding ammonia tank could be quite dangerous.

Hence the need for security. Current industrial computer security is built in layers. A plant computer system is almost certainly connected to the internet. But good security practice is to divide a plant network into several layers and segments. One layer is connected to the internet behind a firewall like any good business security set up. Inside that perimeter, the SCADA system subject to further security controls and is less accessible. Within SCADA, there are often other segments that are further isolated, usually to the point of “air-gapping,” complete physical separation from other computing equipment. This level of security is usually reserved for critical emergency controls that keep the most dangerous processes within safe limits. In theory, these systems are untouchable.

Now the part that made my blood run cold: Triton delivered control of air-gapped critical safety controls to outside hackers.

One of the truths of modern computing is that air-gapped systems can be penetrated. Essentially, the attacker infects the surrounding systems with software that lies in wait, looking for connections to critical hardware controllers, and pounces when it detects a connection. Without the most stringently enforced human security, eventually some hapless technician connects an infected laptop or similar device to the air-gapped system “just for a minute” and the critical system is compromised. Using this technique, the US and Israel compromised and brought down Iranian uranium centrifuges in 2010. Russians brought down Ukrainian power-plants in 2015.

There is always some uncertainty in tracing this kind of hack, but best current opinion is that control of the Saudi refinery was in the hands of a government industrial institute in Moscow for a period in 2017.

Triton appears to have been neutralized. The controller that Triton targeted has been patched. Security practices at the Saudi plant have been revamped. If you are curious, you can read about many of the details, even Python code for detecting Triton here.

I am not likely to purchase industrial gas masks soon. Homeland Security has been helping critical industries to harden their processes (check it out here) and the US still attracts the best computer engineers from everywhere on the planet.

But this is no time to be complacent. Frankly, I have not been impressed with the sophistication of our government in cybersecurity, but I do everything I can to encourage them to do more. The Russians, Chinese, and North Koreans have invested heavily in cyber-warfare. I’ve sent letters to my congressional delegation urging them to fund support for cybersecurity in general and industrial cybersecurity in particular. I urge you to do so also.

Anti-Malware for Apple and Windows

Most Windows users know that anti-malware is necessary, but Apple support implies you don’t need anti-virus or anti-malware installed on your Apple. Well. Mac users do have fewer problems with malware.

A typical Mac-user may go for years without a problem, but that doesn’t mean that Macs are never troubled with malware. From January 1 to January 18, 2019, less than 3 weeks, the Homeland Security central computer vulnerability database recorded 27 new Apple vulnerabilities discovered by security researchers. 7 of these were scored Critical. These are flaws that could be easily exploited to cause serious damage. Macs are not inherently safe.

Macs are less vulnerable

Macs are less vulnerable than Windows for several reasons. First, there are far fewer Apple computers in use than Windows. Hackers follow the money and the money is in hacking Windows. But this is changing. Apple has become more popular, especially among more affluent users, and hackers have noticed.

Second, Apple users tend to stick with installing software from the Apple Store, which Apple polices carefully for security issues. This is safer than the common Windows user practice of downloading software directly from vendors or other web sites.

Finally, Mac OS X, the latest Mac operating system, is based on Unix. Unix (and its most common incarnation, Linux) was designed from the beginning for a multi-user, networked environment where security has always been critical. Windows was originally designed for single user personal computers without network connections. For those early computers, security meant a lock on the front door. Folks worried that a thief would carry off a memory card or the entire machine. Remotely hacking the system was not a thing. That changed when everyone connected to the internet.

Microsoft began to design for security from the ground up about a decade ago. Since then, Microsoft security has made great strides. Windows 10 is much more secure than XP or Vista. Nevertheless, Microsoft is still overcoming years of placing ease of use and rich functionality ahead of security.

The gap is closing

Are Macs still more secure than Windows? I prefer to say that the gap is closing. Also, Mac users may unwittingly transmit email and files that contain Windows malware to Windows computers. Your Mac may be safe, but you could damage your Windows friends. And Windows can transmit Mac malware to Macs. Although Windows and Mac anti-malware products are not interchangeable, most scan for both Windows and Mac issues.

Should Mac users get anti-malware software? If you are a cautious “belt and suspenders” type, you should. If you are a happy-go-lucky risk taker, maybe you can go without and never have a problem, but make no mistake, the risk is there.

Which anti-malware to choose?

For Windows, the simplest and quite adequate solution is to use Windows Defender, which comes installed and activated with Windows 10. Some people prefer third party anti-malware. There are some excellent products. New vulnerabilities appear daily. All the anti-malware developers, including Microsoft, compete vigorously in swatting down the latest malware. It’s a horse race in which the winner changes daily.

Some products to consider for Macs: AVG, Avast, BitDefender, Sophos, MalwareBytes. Other products are good, no products are perfect, but I know and like these. They all have both Mac and Windows versions.

Automatic updates

Be sure to enable automatic updates so your anti-malware is always prepared to thwart the latest attacks. Hacking is an evolving contest with the good guys. You have to keep up. The same applies to operating systems like Windows and Mac OS X and other applications. If you want to be safe, keep them updated.

Most anti-malware products have a free version. In most cases, the free version is as effective as the premium version you pay for, but less convenient. With the free versions, you usually have to start scans yourself instead of letting the system schedule scans for you. The most convenient anti-malware is always on and checking. You won’t even know the best of the products are there, but you pay for the convenience. If you know how, you can write a DIY script yourself to run a free version automatically.

Final caution

Don’t install two anti-malware products at the same time. They can clash and cause trouble. One exception: MalwareBytes is engineered to be compatible with other products. MalwareBytes has an exceptional reputation for cleaning up infected computers after a hack. I’ve heard that techs at Apple Stores use MalwareBytes to clean infected machines.

I run both MalwareBytes and Windows Defender, wear both a belt and suspenders, and always set my emergency brake when I park.

HTTP v. HTTPS

In 2018, you should always use HTTPS (Secure Hypertext Transmission Protocol), right? Well how come Marv Waschke on his sites allows connections using either HTTP or HTTPS? He’s the big advocate for caution on computer networks, isn’t he? So why doesn’t he do what he advocates?

First, allow me to explain what HTTP is and the difference between HTTP and HTTPS. HTTP is a set of rules for exchanging information between a client and a server that is the basis for most communication in the World Wide Web, what you see when you bring up a web browser like Chrome or Firefox.

There are many other protocols that are used on computer networks. HTTP is a very general protocol that can handle many different types of information from straight text to more complex data like sound, photographs, and video. It supports many different kinds of interactions like business transactions on Amazon or live chat. However, a simpler and less flexible protocol will often be faster and more efficient. For example, old fashioned FTP (File Transfer Protocol) will move files from one computer to another with less overhead than HTTP.

In the early and mid-nineties when HTTP was created, the designers quickly recognized that HTTP had significant security flaws. Data is exchanged in clear, unencrypted, text. Anyone with access to the network packet stream can use a packet sniffer like Wireshark to intercept a HTTP data transmission and read it. In the simplest form of HTTP, even passwords are sent in the clear.

Secondly, HTTP offers no guarantee that the sender or receiver is who they say they are. Using HTTP, you may think you are depositing funds into your bank account, but you could just as likely be sending your money to a crook on the other side of the world.

HTTPS was created to close those two gaps. I won’t go into how HTTPS works, but it encrypts data sent over the network and it uses a system of certificates to make it difficult to impersonate web sites. HTTPS is not perfect. The encryption methods used in early versions of the HTTPS standard have been broken, but they are still occasionally used by sites that haven’t kept up with the times. Not long ago, a flaw was found in software used to implement HTTPS (the Heartbleed issue). That flaw has been patched, but you never know when new flaws will be found.

In addition, the certification system is not perfect. Criminals can and do sometimes get certificates. And certificates have to be renewed periodically and not all sites are good about keeping their certificates current.

When HTTPS was first used, both computers and networks were much slower than they are today and therefore HTTPS was considerably slower than HTTP. Consequently, HTTPS was used sparingly. A site like vinemaple.net or marvinwaschke.com where no financial transactions take place and no secrets are exchanged doesn’t need security. The only benefit to using HTTPS is to assure users that they are connecting to the genuine sites, and there isn’t much incentive for anyone to put up a fake site. Since nothing is secret, encrypting doesn’t protect anything.

I currently have both sites set up to use both HTTP and HTTPS. Therefore, no one has to change their old links to my sites and those who would prefer HTTPS security assurances can use HTTPS. Eventually, I’ll phase out the HTTP access, but I’m in no hurry. I encourage you to switch to HTTPS every place you can—it’s a good habit to have. And never perform any kind of financial transaction or convey any data that could be sensitive over HTTP.

Phish Spotting

This morning I was greeted by a spate of phishing emails in my inbox. How did I know? Because the Gmail Team spotted them for me. Google has gotten very good at spotting phishing. I began using Gmail as my main email client yesterday. This morning, the Gmail Team lit up my eyes. I had mixed feelings about Gmail’s unfamiliar interface and online requirement, but their phish spotting performance this morning moved the approval needle a whopping notch in the positive direction. Between 8 and 9 pm yesterday, Google recognized 22 phishing attempts on one of my email accounts. That is close to a denial of service attack, but Google took it in stride.

I’ll ignore the slightly disturbing fact that the phishing started an hour or so after I put the email address into Gmail.

If you are wondering, a Denial Of Service (DOS) attack uses a flood of messages of some kind to try to overwhelm a system. In this case, the flood was phishing email. More often, a DOS attack is on a web service, like Amazon or Google itself, in which a flood of requests for service are sent to the service. The effect is either to slow the service down to the point that legitimate requesters are unable to interact with the service, either because response is so dismal, they turn away, or the service itself fails under the bombardment of requests.

The most difficult type of DOS attack occurs when the attack comes from many different sources at the same time. This is called a Distributed Denial Of Service (DDOS). Often, a DDOS comes from a “botnet,” a collection of surreptitiously invaded computers, often home computers, that are subverted to send out requests at the bidding of the “botmaster.” Often, the point of a phishing attack is to secretly turn your computer into a bot.

Even though Google and other services are effective, don’t become complacent. Automated phishing detection is good, not perfect.

I skimmed over the crop of phishes; they were not well-crafted, mostly warnings of overdue payments from vendors with whom we never deal and notices from online fax services we don’t use. They were riddled with poor grammar and unprofessional formatting. They used slight misspellings in the URLs, like jpmoryan for jpmorgan, to appear that links were to legitimate sources. I’d give you more examples, but I followed best practice and permanently deleted the bogosities. Before I thought of writing about them. Ah well.

If you discipline yourself to look at all incoming email carefully, most phishing attempts are easily weeded out. Look at any invitation to click on a link with suspicion. Check the URL by hovering over the link and looking at the text (usually at the lower left of your display) for anything that looks suspicious like a “.ru” or a misspelled name.

Email attachments, especially zip files, are often treacherous. Don’t open them unless you are very sure of the source. If you know how, start up an isolated virtual machine and open dicey attachments there. If you don’t know what the previous sentence meant, don’t open the attachment. Get competent help if you feel you must open it.

Ask yourself if the message is reasonable. A request from a vendor or service you don’t deal with, for example, is not reasonable. If you think the message might be legit, but you have doubts, pick up the telephone and call their customer service. Don’t use a phone number in the message. Get a number from an independent source, like a secure (https) website that you find using a standard search like Google or Bing. Straight forward caution and common sense takes the day.

Caution, with the help of services like Google’s, will protect you from most phishing, but may not protect you from “spear phishing.” Spear phishing is insidious and seldom automated. Spear phishers study their prey. They can get to information on your preferences and habits collected by advertising services and they can purchase stolen information from criminal sites on the dark web. Or they can look at your public Facebook page, reviews on Amazon, even the book lists that you post at your public library site, to gather details about you, then craft emails that are plausible and hard to detect. For example, a spear phish might take the guise of a letter from a friend suggesting a link about a book you put on your library shelf.

Who wouldn’t be taken in by a friendly gesture like that? Google might spot some discrepancy or a connection to their long list of dangerous sites, but the best spear phishermen strive to stay way ahead of the white hats.

Fortunately, the kind of spear phishing I just described takes far more time, effort, and skill than hackers are willing to expend on random targets. Generally, the criminals rely on sloppy automated scatter gun attacks that only work because they hope for a one in ten thousand catch. However, if you happen to be a high-profile target, like a public figure or a person of interest to a foreign government, and worth the hacker’s effort, you must be cautious indeed. I suggest looking into something like Google’s advanced security program. Benjamin Wittes has a podcast on Google security that you may find worth your time.