X-Rays Can Be Bad For Your Virus's Health
=========================================
By The Sorcerer

1) Introduction, What are X-Rays?
  An old anti-virus (AV) technique that is over looked by most virus writers is X-Ray Detection.
  X-Ray detection is a simple method for detecting encrypted viruses and works on more than 50% of existing encrypted viruses today.
  Have you ever wondered why your new polymorphic, entry point obscuring virus is detected by the AV software?
  The chances are that they have found a X-Ray for your encryption scheme.
  These methods are called X-Rays because they enable the AV software to see the insides of your virus encryption protection without having to emulate your virus.

3) How does X-Ray detection work?
  There are many different X-Ray methods but the following three examples should give you a good understanding of how X-Rays exploit the weaknesses in viruses encryption schemes.

3.1) Simple XOR
  The simple XOR encryption scheme is a very common one used by viruses to hide their contents.
  The simplest version takes every byte to be encrypt / decrypted and XORs it with a key byte.
  E.g.

  Plain Text  : 0A B3 CD 54 
  XOR         : AB AB AB AB
  Cipher Text : A1 18 66 FF

  Now you look at the resulting enciphered code and think ``that is nothing like the original and if I change the key byte each infection then that will be secure.'', but it is not.
  The X-Ray commonly used for this method is simple but a good example of how X-Rays exploit the poorly thought out virus.
  To decode this without emulating the virus the AV software takes the first byte of the suspect cipher text and XORs this with expected Plain Text (e.g. 0A ^ A1) which gives the AV software the key used.
  Then it uses this key to decrypt the rest of the suspected cipher text.  The AV software has then deciphered the virus with out using emulation and with out knowing the key at the start.

3.2) Permuting Keys XOR
  A permuting key XOR is similar to a simple XOR cipher except the key byte is permuted after it has been used to encipher a byte.
  E.g. if we increment the key after it is used, the amount we increase the key by after each use could change after each infection.

  Plain Text  : 0A B3 CD 54 
  XOR         : AB AC AD AE
  Cipher Text : A1 1F 60 FA

  Now as the key is not consistent the method used to X-Ray the simple XOR cipher does not work, but it can be modified to work.
  To X-Ray this example all the AV software has to do is to XOR the first suspected Cipher Text byte with the first plain text byte, which gives it the initial key setting.
  Then the AV software XORs the second suspect cipher text byte with the second plain text value, subtracting from this value the initial key setting previously discovered gives the AV software the degree of increment after each byte has been enciphered.


3.3) Simple Substitution
  Substitution cipher is used less often in viruses but here is how an X-Ray can detect a virus hidden by a simple substitution cipher.
  Substitution ciphers work by replacing bytes with other bytes. E.g.

  Key : A->Z
        B->X
        C->Y

  Plain Text  : ABCBA
  Cipher Text : ZXYXZ

  Now the AV could analyze the suspected cipher text to generate the key, this is a little more complex than with the simple XOR.
  An easier method is to exploit a side effect in the substitution cipher.
  Each value is consistently changed to the value it is mapped to in the key, the positions of each value does not change.
  This enables signatures of patterns to be created e.g.

  Plain Text  : ABCBA
  Cipher Text : ZXYXZ

  Position Signature : [[1,5],[2,4],![1,2,3]]

  The above position signature would detect the plain text from the cipher text.
  The position signature shown states that positions 1 and 5 should match (Plain Text A's), positions 2 and 4 should match (Plain Text B's) and positions 1, 2 and 3 should not match (Plain Text A, B and C).

4) How to hide from X-Ray Detection
  So having seen how powerful the X-Ray detection methods can be how can viruses hide from them?
  The interesting thing to note is that all the X-Ray methods exploit some holes in the viruses encryption methods.
  In Cryptanalysis terms the viruses encryption methods susceptible to X-Ray attacks are all broken when used with known plain text attack.
  So Metamorphism of the code inside the encrypted section of a viruses would help defend it against X-Ray detections as the X-Ray no longer knows what the plain text is.

  Substitution Ciphers if used should be polyalphabetic, that is to say they should not use one key but cycle through a set of keys.
  Also the order the sets of keys are used in should change between infections.
  Transpositional ciphers should be considered as well to increase the strength of the substitution cipher.
  A transpositional cipher is one where the positions of values are swapped, e.g.

  Plain Text  : ABCDEF 
  Key         : 1<->2
                4<->3
                5<->6
  Cipher Text : BADCFE

  Better pseudo random number generators (PRNG) will also help improve the strength of the encryption methods in use.
  Instead of increasing the value of the key by a constant amount using a good PRNG will stop the key increment being predicted.

  Combination of ciphers can also improve the security of the cipher.
  It is always worth considering what can be done in one pass and what will need multiple passes.
  A substitution cipher and an XOR cipher can be combined in a single pass while a transpositional cipher and an XOR cipher will need a second pass.

  Multiple rounds of encryption can also improve the strength of the encryption.
  Instead of using a polyalphabetical substitution cipher with a transpositional cipher once, use then many times.
  Generally the more rounds used the stronger the encryption.

  Finally the key stored in the virus should not be too distinct, otherwise the AV software may be able to find the key rather than have to guess.
  E.G. for the simple substitution cipher the stored key will consist of block of 256 bytes and each value between 0 and 255 will only appear once in the block.
  This is very distinct and as such will is not hard to find.

5) Conclusion
  Improving the encryption routine makes it harder for AV software to detect a virus.
  Weak encryption schemes should not be used.  Time spent on improving a viruses encryption scheme will help the virus survive longer.
  Don't forget though that emulation is the AV software's next step, but an encryption routine that has many rounds can help against that as well (How long will it emulate a program to find a virus).