TLS is divided into two layers: recording layer and handshake layer, in which the handshake layer contains four sub-protocols: handshake protocol, password specification change protocol, application data protocol and alarm protocol.
You only need to configure the browser and server settings to open TLS, so that HTTPS and TLS can be highly decoupled, detachable, cooperative and independent from the upper advanced application layer protocols.
The function realization of TLS/SSL mainly depends on three basic algorithms: Hash function hash, symmetric encryption and asymmetric encryption. Among them, asymmetric encryption is used to realize identity authentication and key negotiation, and symmetric encryption algorithm uses negotiated key to encrypt data, and verifies the integrity of information based on hash function.
The basic working mode of TLS is that the client uses asymmetric encryption to communicate with the server, realizes identity authentication and negotiates the key used for symmetric encryption, and then the symmetric encryption algorithm encrypts the information and information digest with the negotiated key. Different nodes use different symmetric keys, thus ensuring that information can only be obtained by both parties.
For example, in the HTTPS protocol, when the client sends a request, the server will send the public key to the client. The client will generate a key after authentication, and then encrypt it with the public key and send it to the server (asymmetric encryption). Both parties will generate a negotiation key (symmetric key) during the TLS handshake, and then establish an encrypted connection after success. In the communication process, the client encrypts the request data with the negotiation key and sends it, and the server decrypts it with the negotiation key, and the response also uses the same negotiation key. Subsequent communication uses symmetric encryption because symmetric encryption and decryption speed is fast, while asymmetric encryption can ensure the effectiveness of encryption when shaking hands, but the process is complicated and the calculation amount is relatively large.
The recording protocol is responsible for transmitting all the underlying messages exchanged on the connection and can configure encryption. Each TLS record begins with a short title. The header contains the type (or sub-protocol), protocol version and length of the recorded content. The original message is segmented (or merged), compressed, added with an authentication code and encrypted into the data part of the TLS record.
The recording layer divides information blocks into TLSPlaintext records, which carry data blocks of 2 14 bytes (16KB) or smaller.
Recording protocol transmits opaque data buffers submitted to it by other protocol layers. If the buffer exceeds the recording length limit (2 14), the recording protocol will cut it into smaller fragments. The reverse is also possible, and small buffers belonging to the same sub-protocol can also be combined into a single record.
The compression algorithm converts TLSPlaintext structure into TLSCompressed structure. If the definition of CompressionMethod is null, there is no compression.
The bulk encryption algorithm transforms the TLSCompressed.fragment structure into the stream TLSCiphertext.fragment structure.
The MAC generation method is as follows:
Seq_num (serial number of record), hash (hash algorithm specified by securityparameters. MAC _ algorithm).
Block encryption (such as RC2 or DES), which transforms the TLSCompressed.fragment structure into a block TLSCiphertext.fragment structure.
Padding: The added padding forces the plaintext length to be an integer multiple of the block length of the block cipher. The padding length can be any length within 255 bytes, as long as TLSCiphertext.length is an integer multiple of the block length. According to the analysis of the length of exchanged information, the length greater than the required value can prevent protocol attacks. Each uint8 in the padding data vector must be padded with a padding length value (padding_length).
Padding_length: the padding length should be such that the total size of the GenericBlockCipher structure is a multiple of the encryption block length. Legal values range from 0 to 255, inclusive. This length specifies the length of the padding field, not the length of the padding_length field itself.
The data length (TLSCiphertext.length) of the encrypted block is the sum of TLSCompressed.length, CipherSpec.hash_size and padding_length plus 1.
Encryption and MAC functions convert TLS compression structure into TLSCiphertext. The recorded MAC also includes a sequence number so that lost, extra or duplicate messages can be detected.
Recording protocol needs an algorithm to generate keys, IV and MAC secrets from the security parameters provided by handshake protocol.
Master key: a 48-byte key shared by both parties.
Client Random: a 32-byte value provided by the client.
Server Random Number: a 32-byte value provided by the server.
Handshaking is the most complicated part of TLS protocol. In this process, the communication parties negotiate the connection parameters and complete the identity verification. Depending on the functions used, the whole process usually needs to exchange 6~ 10 messages. Depending on the configuration and supported protocol extensions, there may be many variants of the exchange process. In use, we can often observe the following three processes: (1) completing the handshake authentication server; (2) Short handshakes used in previous sessions; (3) Handshake for authentication between client and server.
The header information of handshake protocol message includes message type (1 byte) and length (3 bytes), and the rest information depends on the message type:
Every TLS connection will start with a handshake. If the client has not established a session with the server before, the two parties will perform a complete handshake process to negotiate a TLS session. During the handshake, the client and the server will take the following four main steps:
The following are the most common handshake rules, which do not need to verify the identity of the client but need to verify the identity of the server:
This message conveys the functions and preferences of the client to the server.
Yes, the connection parameters selected by the server are returned to the client.
The structure of this message is similar to that of ClientHello, except that each field contains only one option, including the random_S parameter of the server (used for subsequent key negotiation). The server does not need to support the best version supported by the client. If the server does not support the same version as the client, it can provide some other versions, hoping that the client can accept it.
The cipher suite in the figure is an encryption suite used for subsequent key agreement and authentication. The key exchange and signature algorithm chosen here is ECDHE_RSA, and the symmetric encryption algorithm is AES-GCM, which will be mentioned later.
Another point is that TLS compression is turned off by default, because criminal attacks will use TLS compression to recover encrypted authentication cookie and realize session hijacking. However, it is generally unfavorable to configure gzip and other content to compress TLS fragments and occupy additional resources, so TLS compression is generally turned off.
A typical certificate message is used to carry the server X.509 certificate chain.
The server must ensure that the certificate it sends is consistent with the selected algorithm group. For example, the public key algorithm must match the algorithm used in the suite. In addition, some key exchange algorithms rely on specific data embedded in the certificate and require the certificate to be signed by an algorithm supported by the client. All these indicate that the server needs to configure multiple certificates (each certificate may be equipped with a different certificate chain).
The certificate message is optional because not all suites use authentication, and not all authentication methods require certificates. In addition, although messages use X.509 certificates by default, they can also carry other forms of symbols; Some packages rely on PGP keys.
Carrying extra data needed for key exchange. ServerKeyExchange is optional, and the message content will be different for different negotiation algorithm suites. In some cases, such as using RSA algorithm, the server does not need to send this message.
The server sends the ServerKeyExchange only when the server certificate message (the certificate message mentioned above) does not contain enough data to allow the client to exchange the pre-master key.
For example, in the handshake process based on DH algorithm, a ServerKeyExchange message with DH parameters needs to be sent separately:
Indicates that the server has sent all expected handshake messages. After that, the server will wait for the client to send the message.
The client verifies the validity of the certificate, and then carries out subsequent communication after passing the verification, or prompts and operates according to different error situations. The contents of validity verification include the following:
According to the content of PKI system, the signature of the certificate sent by the peer is encrypted with the private key of CA. After receiving the certificate, first read the relevant plaintext information in the certificate, use the same hash function to calculate the information digest, and then use the public key of the corresponding CA to decrypt the signature data. By comparing the information digest of the certificate, the validity of the certificate can be confirmed. Then check the revocation of the certificate, etc.
After verifying the validity, the client calculates the pre-owner of the generated random number, encrypts it with the certificate public key, and sends it to the server for key exchange together with all the information provided by the client. The message is influenced by the negotiated cipher suite, and its content varies with different negotiated cipher suites.
At this point, the client has obtained all the information needed to calculate the negotiation key: two plaintext random numbers random_C and random_S and the Pre-master generated by its own calculation, and then obtained the negotiation key (for later message encryption).
ECDHE algorithm is used in the figure, and ClientKeyExchange passes the client parameters of DH algorithm. If RSA algorithm is used, the encrypted pre-master key should be passed here.
Notify the server that the subsequent communication adopts the negotiated communication key and encryption algorithm for encrypted communication.
The completion message means that the handshake has been completed. The content of the message will be encrypted so that both parties can exchange the data needed to verify the integrity of the whole handshake safely.
This message contains a verify_data field whose value is the hash value of all messages during the handshake. These messages are arranged at both ends of the connection in the order in which they are seen, and the hash is calculated with the negotiated master key (enc_key). This process is completed by pseudo-random function (PRF), which can generate any number of pseudo-random data.
The calculation method of both ends is the same, but different labels (finished_label) are used: the client is completed by the client and the server is completed by the server.
Because the completed messages are encrypted and their integrity is guaranteed by the negotiated MAC algorithm, the active network attacker cannot change the handshake message and forge the value of vertify_data. In the version of TLS 1.2, the length of the completion message defaults to 12 bytes (96 bits), and the cipher suite allows a longer length. Prior to this, all versions except SSL 3 used fixed-length messages of 12 bytes.
The server decrypts the encrypted Pre-master data with the private key, and based on the two plaintext random numbers random_C and random_S exchanged before, the negotiation key is also calculated: enc _ key = PRF (pre _ master, "master secret", random _ c+random _ s);
Calculate the hash values of all previous messages, and then decrypt verify_data_C sent by the client with the negotiation key to verify the correctness of the messages;
After the authentication of the server passes, the server also sends change_cipher_spec to inform the client to use the agreed key and algorithm for encrypted communication in the subsequent communication (a new session ticket is added in the figure, which will be explained in the session recovery);
The server also generates a piece of data (verify_data_S) by combining all the current communication parameter information, encrypts it with the negotiation key session secret (enc_key) and sends it to the client;
The client calculates the hash value of all the received information, decrypts verify_data_S with the negotiated key, verifies the data and key sent by the server, and completes the handshake if the verification passes;
Use the negotiated key to start encrypted communication through the algorithm.
HTTPS provides three functions through TLS layer and certificate mechanism: content encryption, authentication and data integrity. In the process of encryption, two algorithms, asymmetric key exchange and symmetric content encryption, are needed.
Symmetric content has very high encryption strength and fast encryption and decryption speed, but it can't safely generate and save keys. In TLS protocol, the last application data is transmitted after symmetric encryption, and the symmetric negotiation key (enc_key above) used in transmission is exchanged through asymmetric key in handshake phase. Common AES-GCM and ChaCha20-Poly 1305 are symmetric encryption algorithms.
Asymmetric key exchange can generate symmetric encryption keys only known by both parties in insecure data channels. At present, the most commonly used key exchange algorithms are RSA and ECDHE.
RSA has a long history and good support, but it does not support perfect forward security-PFS. ECDHE is a DH(Diffie-Hellman) algorithm using ECC (Elliptic Curve), which has fast calculation speed and supports PFS.
In the part of PKI system, it is explained that only asymmetric key exchange can not resist MITM attack, so it is necessary to introduce PKI system certificate for identity verification, in which the public key generated by asymmetric encryption on the server side will be put in the certificate and transmitted to the client.
In RSA key exchange, the browser uses the RSA public key provided by the certificate to encrypt related information. If the server can decrypt it, it means that the server has a private key corresponding to the public key, and at the same time, the key needed for symmetric encryption can be calculated. Key exchange and server authentication are combined.
In ECDH key exchange, the server uses the private key (RSA or ECDSA) to sign the related information. If the browser can verify the signature with the certificate public key, it means that the server does have the corresponding private key, thus completing the server authentication. The key exchange is completed by sending DH parameters separately, and the key exchange and server authentication are completely separated.
The algorithms that can be used for ECDHE digital signature mainly include RSA and ECD sa- elliptic curve digital signature algorithm, that is, there are three mainstream key exchange+signature schemes at present:
For example, the encryption suite used in my website is ECDHE_RSA, and we can see that the digital signature algorithm is sha256 hash plus RSA encryption. In the PKI system, it is said that the signature is generated by encrypting the hash value of the server information abstract.
Certificates with built-in ECDSA public key are generally called ECC certificates, and certificates with built-in RSA public key are RSA certificates. Because the 256-bit ECC key is equivalent to the 3072-bit RSA key in security, the ECC certificate is smaller than the RSA certificate and the ECC operation speed is faster. ECDHE key exchange +ECDSA digital signature is the best encryption suite at present.
The above content comes from this article: Start using ECC certificates.
More details about ECC certificates can be found in the document: ECC cipher suite for TLS-RFC4492.
The handshake process of key exchange using RSA is basically the same as described above, except that there is no ServerKeyExchange message, in which key negotiation involves three parameters (client random number _ c, server random number _ s, Premaster secret).
Among them, the first two random numbers and the algorithm used in the negotiation are easy to obtain in plaintext, and the last pre-master secret will be encrypted with the public key provided by the server and transmitted to the server (key exchange). If this pre-master key is intercepted and cracked, the negotiation key can also be cracked.
RSA algorithm can be found in wiki and RSA algorithm principle (II)-Ruan Yifeng.
The core idea of RSA algorithm is to use the computational complexity of maximum integer factorization.
When DH(Diffie-Hellman) algorithm is used for key exchange, both parties can calculate the pre-master key by exchanging their respective DH parameters (sending server parameters in ServerKeyExchange and sending client parameters in ClientKeyExchange) without passing the pre-master key.
The handshake process of DH is as follows. The general process is similar to RSA, and the figure only shows how to generate the pre-master key:
The server signs the client random number _ c, the server random number _ s and the server DH parameter Server params through the private key to generate a signature, and then sends the server DH parameter and signature in the ServerKeyExchange message;
After receiving it, the client decrypts the verification signature with the public key given by the server, and sends the client DH parameter Client params in the ClientKeyExchange message;
After the server receives it, both parties have these two parameters, and then use these two parameters to generate the Premaster secret, and then the steps of negotiating the key are basically the same as RSA.
About how DH algorithm generates pre-master key, it is recommended to look at Wiki and brief Diffie-Hellman handshake.
Its core idea is to use the computational complexity of discrete logarithm problem.
The algorithm flow can be abstracted as the following figure:
The two parties agreed in advance on a pair of P g values (public), while Alice has a private number A (private, corresponding to a private key) and Bob has a private number B (private, corresponding to a private key).
For Alice and Bob, the final same key can be obtained through the public key parameters sent by the other party and the private key in their hands.
And the third party knows about P g A B at most, so it is difficult to get the private key and the final key. Of course, the premise is that a b P is large enough (there are several commonly used large prime numbers in RFC3526 document), otherwise it is possible to try out the answer with brute force. As for g, a smaller value is generally enough.
The following figure shows the contents sent by the actual DH handshake:
It can be seen that the parameters sent by both parties carry a public key value, which corresponds to A and B above.
In addition, the actual encryption suite is elliptic curve DH key exchange (ECDH). Using elliptic curve encryption to establish a public key and private key pair can further enhance the security of DH, because it is much more difficult to solve the problem of discrete logarithm of elliptic curve than factorization at present, and the key length used by ECC is much shorter than that of RSA key (RSA key needs more than 2048 bits to ensure security, while ECC key needs 256 bits).
Regarding ECC, I recommend an introductory book on ECC-Original Translation.
Although you can choose to authenticate either side, almost everyone has enabled authentication to the server. If the suite selected by the server is not anonymous, you need to keep your certificate in the certificate message.
Instead, the server requests authentication of the client by sending a CertificateRequest message. All acceptable client certificates are listed in the message. In response, the client sends its own certificate message (using the same format as the certificate sent by the server) and attaches the certificate. After that, the client sends a CertificateVerify message to prove that it has the corresponding private key.
Only authenticated servers are allowed to request client authentication. Therefore, this option is also called mutual authentication.
Sent in the process of ServerHello, requesting to authenticate the client, and transmitting the public key and signature algorithm of the certificate it accepts to the client.
It can also choose to send a list of certification authorities it accepts, which are all represented by their distinguished names:
Publish in the process of ClientKeyExchange to prove that the private key you have matches the public key in the client certificate you sent before. This message contains the signatures of all handshake messages up to this step:
The original session recovery mechanism is that when the fully negotiated connection is disconnected, both the client and the server will save the security parameters of the session for a period of time. The server that wants to use session recovery assigns a unique identification to the session, which is called ID (session ID. The server sends the session ID back to the client in a ServerHello message.
A client wishing to resume an earlier session puts the appropriate session ID in the ClientHello message and submits it. If the server agrees to resume the session, it will return the same session ID in the ServerHello message, then use the previously negotiated master key to generate a new set of keys, and then switch to encryption mode to send the completed message.
After receiving the message that the session has been resumed, the client will do the same. The result is that only one network round trip is needed to shake hands.
Session ID is supported by the server, which is a standard field in the protocol, so basically all servers support it. The server saves the session ID and the negotiated communication information, which will occupy more server resources.
The solution to replace server session caching and recovery is to use session tickets. In this way, the message flow is the same as the server session cache, except that all state is saved in the client (similar to the principle of HTTP Cookie).
The idea is that the server takes out all its session data (state) and encrypts it (the key is only known to the server), and then sends it back to the client in the form of a ticket. In the next connection, when the client resumes the session, it will submit the ticket back to the server using the encrypted information in the extended field session_ticket of ClientHello, and the server will check the integrity of the ticket, decrypt its contents, and then resume the session using the information in it.
This method may make it easier to expand the server cluster, because if you don't use this method, you need to synchronize sessions between nodes serving the cluster.
Session ticket needs the support of server and client, which belongs to the extended field and occupies little server resources.