Last updated: Tue Sep 21 10:20:06 2021
As of recent times, more people are raising support questions with regards to AntiVirus warnings on the downloads of CRiSP software. Whilst adequate steps are made, to ensure that both the website itself, and the development machines are in fact, free of malware, these reports are raised.
On examination of some of these reports, and investigating what the AV tools actually do, alas, they are generating false reports. Now, in the absence of absolute proof of an issue, it is easy to be disingenous and relate that these software downloads are trouble-free, but I wanted to take the time to educate a little on the investigation and what I have found.
AV tools have a hard time, trying to detect the many variants of malware. Malware can take control of your machine, and do any of the things that are widely reported in the press. Such attacks involve modifying or replacing parts of an application or the entire application itself. In investigating this, and trying various things to ensure no false reports, it has been increasingly difficult to do this, for two reasons:
Code signing is equivalent to placing a time-limited password inside the application, (for those that know, think of RSA and public/private keys). Unfortunately one aspect of this means that the executable you download, would eventually expire, and no ability to re-use such software, should Foxtrot Systems cease to be (or the relevant vendor).
Code-signing involves signing the code. What this means, is that, using a private key, some bytes are stored in the file, and the operating system can then validate if its correct. This is similar to how "https" websites work. If the OS does not like what it sees, it will refuse to run.
Code signing also involves paying for a certificate from the relevant vendor (eg Microsoft or Apple). This adds to the cost & complexity of the product, but also, adds to the need to continuously subscribe to the service, even as other factors may make this situation irrelevant. Code-signing is an important aspect of Windows 10. It didn't exist, to my knowledge in Windows XP, Vista, Windows 7. So, if you use older platforms, and want the latest code releases, you are not going to find it easy. In the future, the existing mechanisms supported by the OS may change.
A problem with certificate signing is that the signing is done for valid time period. This could be as short as 12 months, or as long as 30-40 years. However, the issuing party (eg Microsoft, Apple, etc) can revoke these certificates, should there be concern on their authenticity. This means one day, the software may stop working, despite not reaching its time limit, and again, you may be blocked from using any latest release, assuming the vendor is still in business!
All of this is unfortunate: it means the steps taken to protect end users, require constant attention and updates, solely for the meta-management of certificates.
AV False Positives
There are many forms of malware, and the AV product vendors have built up databases to catalog these, and to detect them. Some may be as simple as detecting a byte-sequence in a file; others may resort for looking at tell-tale signs of tampering of a file. For example, the Windows EXE file format is a series of sections; if you append rubbish to the end of a EXE file, the EXE is still well-formed and will run correctly. But existence of extraneous bytes at the end of the file, might signal malware presence. Another form of AV detection is to actually decode an application, and look for signs of known code injection attacks. Of course, the AV tool cannot run an application through every code path. (For a start this requires enormous amounts of processing power, and, very likely would never complete - often known as the "Turing Halting Problem"). But they can do some great work to detect payloads using all sorts of 'secret sauce'. It is in the AV vendors interest, to build up large databases to cover every known virus, and highlight how many items it can detect. Many of these may be past their sell by date. And worse, they may falsely accuse an application of an issue, incorrectly. It is nearly impossible to get an AV vendor to recognize a false-positive.
This website provides a cloud based service, allowing uploads of files, and it will report any AV issues, based on using various vendors' AV products. It is an excellent site in many respects, and I encourage people to take copies of CRiSP and run it through the site.
Availability of sites like this are a useful end user service. They are also an excellent service to every malware writer. If malware can successfully be disguised and get a clear sign off from such a service, there is much less chance of real world detection. In fact, malware writers can write scripts to keep mutating a payload, until it in fact, passes.
Alas, this means that for non-malware writers, they are in the same boat. Proving a large application is virus free is near impossible. Even if the source code was available, and you compiled it yourself, you would still get AV false positives. And no end user has the time to do this.
CRiSP's solution to this
CRiSP itself is made available as an installer from the downloads section of this website. It has always been expected, that one day, the webserver would be attacked, and mutant variants of software would be silently available to end users.
In the near 30 years of existence of this service, no signs of tampering or breaking in have ever been detected. Of course, the service is attacked daily to try and break in, but no evidence exists of success. Despite this, the software has always been provided with a checksum. And duplicate copies of files are available, to prove whether the web service has been attacked.
This area is complex. The checksums provide a weak form of protection, because, any attacker who can update the binaries, can update the recorded checksus, but methods are available to detect even that.
In addition, the software installer has builtin checksum capability. Should the software be tampered with the installer will detect this.
A problem with this scenario of attacker/defender (a.k.a cat and mouse), is that should an attacker be good enough to silently update the software downloads, then they likely are good enough to update the installer/detection mechanisms.
To help mitigate this, and, get a flying-colors from the AV tools, the crisp installer actually wraps the install payload with another installer - effectively a russian doll approach, and each layer is validated against a checksum.
The actual payload is no longer a gzipped binary file; it is now encoded in plain text, and automatically extracted on installation. In theory, this should have lead to zero detections by any AV tool.
The virustotal website, above, and Windows Defender, may still highlight issues in the software download. In fact, for Windows and MacOS, the default is to reject anything which is not code-signed by themselves. Since CRiSP is not code-signed, then they will generate a warning. End users can simply choose to accept the risk of installing this software.
For virustotal, despite 99% of the download being plain text, it can still generate reports. The implication is the installer itself is infected, which is hard to believe, given its simplicity and lack of dependencies. Unfortunately, whilst it can report that there are signs of specific virus signatures, it is impossible to determine what it is noticing, except by using mutation techniques - add noise to the payload, until the problem goes away. (This has been done to a certain extent, but requires more work).
Hopefully this document will help individuals decide for themselves whether to trust the software. It is nearly impossible to be 100% certain whether this software, or any software (even with code signing) is untainted, but this, unfortunately is a factor of the world we live in.