windows authentication protocols
All Blog

The Windows operating system implements a default set of authentication protocols, including Kerberos, NTLM, Transport Layer Security/Secure Sockets Layer (TLS/SSL), and Digest, as part of an extensible architecture. In this post, we’ll be looking at LM & NTLM hashes and Kerberos.


LAN Manager, or LM, is an authentication protocol designed (at its time) to maximize password security in a Windows-based environment. The LM protocol was first used in Microsoft’s LAN Manager product a very long time ago and is still the authentication protocol of choice for older operating systems, such as Windows 95 and Windows NT 3.51 and earlier. Later, when Windows NT was introduced, LM was enhanced and renamed the NTLM authentication protocol. Although NTLM has been around for a long time, it’s still a basically good authentication protocol, and it is the native network authentication protocol of Windows NT 4.0 and earlier operating systems.

Functioning of Lan Manager hash:

  • DES encryption of a fixed value using a password as a key
  • Passwords should not exceed 14 characters
  • Every 7 characters in a password are encrypted separately
  • Upper/lower cases are not distinguished
  • Total number of passwords made of alphanumerics and symbols:about 7.5 trillion
lm hash
Lan manager (LM) Authentication

LM response

  • LM Hash

Passwords made up of alphanumerics and symbols: about 7.5 trillion

  • Attackers can instantaneously determine if a password exceeds 7 characters or not.
  • Does not protect against precomputed dictionary attacks

Server sending a fixed challenge


NTLM is Microsoft’s mythological legacy authentication protocol. Although new and better authentication protocols have already been developed, NTLM is still very much in use – even the most recent Windows versions support NTLM, and its use is still required when deploying Active Directory.

While better solutions are already in use, the obvious question is why the NTLM protocol is still here? Basically, because NTLM is a legacy protocol, it is very hard to disable without causing damage to production systems. The challenge starts with determining which machines require the use of this function and which don’t. After mapping the usage, it is hard to determine how to move from NTLM usage to a more secure authentication protocol without breaking anything.

NTLM has two versions – NTLMv1 and NTLMv2. NTLMv2 suppose to offer better security than its previous version, and to some extent it does provides better defense against relay and brute force attacks, but does not completely block them.


The NTLMv1 protocol uses an NT Hash or KM hash (depending on its configuration), in a challenge/response exchange between the server and the client. The NTLM authentication flow is as follows:

  1. The client machine sends a request to connect to the server.
  2. The server generates a random nonce to be encrypted by the client.
  3. The client machine encrypts the nonce with the password hash to prove knowledge of the password.
  4. The server validates the user’s identity by ensuring that the challenge was indeed created with the correct user/password. It does this either by using data from its own SAM database or by forwarding challenge-response pairs for validation in the domain controller.

Both the hashes produce 16-byte quantities. Five bytes of zeros are appended to obtain 21 bytes. The 21 bytes are separated into three 7-byte (56-bit) quantities. Each of these 56-bit quantities is used as a key to DES encrypt the 64-bit challenge. The three encryptions of the challenge are reunited to form the 24-byte response. Both the response using the LM hash and the NT hash is returned as the response, but this is configurable.

C = 8-byte server challenge, random
K1 | K2 | K3 = NTLM-Hash | 5-bytes-0
response = DES(K1,C) | DES(K2,C) | DES(K3,C)
ntlm v1


  • Does not protect against pre-computed dictionary attacks

                                   – Server sending a fixed challenge

  • Concerns – DES key space is not large enough:

                                   256 = About 70 quadrillion


NTLMv2, introduced in Windows NT 4.0 SP4, is a challenge-response authentication protocol. It is intended as a cryptographically strengthened replacement for NTLMv1.

NTLM version 2 (NTLMv2), which was introduced in Windows NT 4.0 SP4 (and natively supported in Windows 2000), enhances NTLM security by hardening the protocol against many spoofing attacks and adding the ability for a server to authenticate to the client.

NTLMv2 sends two responses to an 8-byte server challenge. Each response contains a 16-byte HMAC-MD5 hash of the server challenge, a fully/partially randomly generated client challenge, and an HMAC-MD5 hash of the user’s password and other identifying information. The two responses differ in the format of the client challenge. The shorter response uses an 8-byte random value for this challenge. To verify the response, the server must receive as part of the response the client challenge. For this shorter response, the 8-byte client challenge appended to the 16-byte response makes a 24-byte package which is consistent with the 24-byte response format of the previous NTLMv1 protocol. In certain non-official documentation (e.g. DCE/RPC Over SMB, Leighton) this response is termed LMv2.

The second response sent by NTLMv2 uses a variable-length client challenge which includes (1) the current time in NT Time format, (2) an 8-byte random value (CC2 in the box below), (3) the domain name, and (4) some standard format stuff. The response must include a copy of this client challenge, and is, therefore, variable in length. In non-official documentation, this response is termed NTv2.

Both LMv2 and NTv2 hash the client and server challenge with the NT hash of the user’s password and other identifying information. The exact formula is, to begin with, the NT hash, which is stored in the SAM or AD, and continues to hash in, using HMAC-MD5, the username, and domain name. In the box below, X stands for the fixed contents of a formatting field.

SC = 8-byte server challenge, random
CC = 8-byte client challenge, random
CC* = (X, time, CC2, domain name)
v2-Hash = HMAC-MD5(NT-Hash, user name, domain name)
LMv2 = HMAC-MD5(v2-Hash, SC, CC)
NTv2 = HMAC-MD5(v2-Hash, SC, CC*)
response = LMv2 | CC | NTv2 | CC*
ntlm v2


Weak cryptography:

The NTLM cryptography scheme is relatively weak, making it relatively easy to crack hashes and derive plaintext passwords. It’s easy enough for standard hardware to be able to crack an 8-character password in less than a day. This is for three main reasons:

  1. The hash is based on MD4, which is relatively weak.
  2. The hash is saved unsalted in a machine’s memory before it is salted and sent over the wire.
  3. A user must respond to a challenge from the target, which exposes the password to offline cracking. This prevents offline Relay attacks.

No mutual authentication:

This flaw exposes the protocol to a man-in-the-middle (MITM) attack. When a client communicates with a server, it does not validate the server’s identity (this is known as one-way authentication). A malicious actor with MITM capabilities can send malicious data to the client while impersonating the server.


Kerberos is a computer network authentication protocol that works based on tickets to allow nodes communicating over a non-secure network to prove their identity to one another in a secure manner. Its designers aimed it primarily at a client–server model and it provides mutual authentication—both the user and the server verify each other’s identity. Kerberos protocol messages are protected against eavesdropping and replay attacks.

It builds on symmetric key cryptography and requires a trusted third party, and optionally may use public-key cryptography during certain phases of authentication. Kerberos uses UDP port 88 by default.

So basically provides a centralized authentication server whose function is to authenticate users to servers and servers to users. In Kerberos Authentication server and database is used for client authentication. Kerberos runs as a third-party trusted server known as the Key Distribution Center (KDC). Each user and service on the network is a principal.

The main components of Kerberos are:

  • Authentication Server (AS):
    The Authentication Server performs the initial authentication and ticket for Ticket Granting Service.
  • Database:
    The Authentication Server verifies the access rights of users in the database.
  • Ticket Granting Server (TGS):
    The Ticket Granting Server issues the ticket for the Server
  1. User logs on and requests services on the host. Thus user requests for ticket-granting-service.
  2. Authentication Server verifies user’s access right using database and then gives ticket-granting ticket and session key. Results are encrypted using the Password of the user.
  3. The decryption of the message is done using the password then send the ticket to Ticket Granting Server. The Ticket contains authenticators like user names and network address.
  4. Ticket Granting Server decrypts the ticket sent by the User and the authenticator verifies the request then creates the ticket for requesting services from the Server.
  5. The user sends the Ticket and Authenticator to the Server.
  6. The server verifies the Ticket and authenticators then generate access to the service. After this User can access the services.

Even with its impenetrable security, it can still be vulnerable to attack if not configured properly. Below is the list of few already known vulnerabilities to keeping in mind while configuring your system.

CVE-2009-4212 Multiple integer underflows in the (1) AES and (2) RC4 decryption functionality in the crypto library in MIT Kerberos 5 (aka krb5) 1.3 through 1.6.3, and 1.7 before 1.7.1, allow remote attackers to cause a denial of service (daemon crash) or possibly execute arbitrary code by providing ciphertext with a length that is too short to be valid.
CVE-2009-0846The asn1_decode_generaltime function in lib/krb5/asn.1/asn1_decode.c in the ASN.1 GeneralizedTime decoder in MIT Kerberos 5 (aka krb5) before 1.6.4 allows remote attackers to cause a denial of service (daemon crash) or possibly execute arbitrary code via vectors involving an invalid DER encoding that triggers a free of an uninitialized pointer.
CVE-2004-0523Multiple buffer-overflows in krb5_aname_to_localname for MIT Kerberos 5 (krb5) 1.3.3 and earlier allow remote attackers to execute arbitrary code as root.
CVE-2003-0139Certain weaknesses in the implementation of version 4 of the Kerberos protocol (krb4) in the krb5 distribution, when triple-DES keys are used to key krb4 services, allow an attacker to create krb4 tickets for unauthorized principals using a cut-and-paste attack and “ticket splicing.”
CVE-2003-0138Version 4 of the Kerberos protocol (krb4), as used in Heimdal and other packages, allows an attacker to impersonate any principal in a realm via a chosen-plaintext attack.
CVE-2001-0554Buffer overflow in BSD-based telnetd telnet daemon on various operating systems allows remote attackers to execute arbitrary commands via a set of options including AYT (Are You There), which is not properly handled by the telrcv function.
CVE-2000-0392Buffer overflow in ksu in Kerberos 5 allows local users to gain root privileges.
CVE-2000-0391Buffer overflow in krshd in Kerberos 5 allows remote attackers to gain root privileges.
CVE-2000-0390Buffer overflow in krb425_conv_principal function in Kerberos 5 allows remote attackers to gain root privileges.
CVE-2000-0389Buffer overflow in krb_rd_req function in Kerberos 4 and 5 allows remote attackers to gain root privileges.

Leave a Reply

Your email address will not be published.