C++ Convert RSA System from old to new

[X X X]

Hello,

I have tried to convert the RSA system on my server (TFS 1.2) from the "old" style of having the prime1 and prime2 hard-coded in the sources to the "new" style, with a key.pem file, but I can't get it to work. Here is what I have done so far:

I replaced these lines in my otserv.cpp:

//set RSA key
    const char* p("14299623962416399520070177382898895550795403345466153217470516082934737582776038882967213386204600674145392845853859217990626450972452084065728686565928113");
    const char* q("7630979195970404721891201847792002125535401292779123937207447574596692788513647179235335529307251350570728407373705564708871762033017096809910315212884101");
    g_RSA.setKey(p, q);

with this:

//set RSA key
    try {
        g_RSA.loadPEM("key.pem");
    } catch(const std::exception& e) {
        startupErrorMessage(e.what());
        return;
    }

Then, I replaced my RSA.cpp file, which used to look like this:

#include "otpch.h"

#include "rsa.h"

RSA::RSA()
{
    mpz_init(n);
    mpz_init2(d, 1024);
}

RSA::~RSA()
{
    mpz_clear(n);
    mpz_clear(d);
}

void RSA::setKey(const char* pString, const char* qString)
{
    mpz_t p, q, e;
    mpz_init2(p, 1024);
    mpz_init2(q, 1024);
    mpz_init(e);

    mpz_set_str(p, pString, 10);
    mpz_set_str(q, qString, 10);

    // e = 65537
    mpz_set_ui(e, 65537);

    // n = p * q
    mpz_mul(n, p, q);

    mpz_t p_1, q_1, pq_1;
    mpz_init2(p_1, 1024);
    mpz_init2(q_1, 1024);
    mpz_init2(pq_1, 1024);

    mpz_sub_ui(p_1, p, 1);
    mpz_sub_ui(q_1, q, 1);

    // pq_1 = (p -1)(q - 1)
    mpz_mul(pq_1, p_1, q_1);

    // d = e^-1 mod (p - 1)(q - 1)
    mpz_invert(d, e, pq_1);

    mpz_clear(p_1);
    mpz_clear(q_1);
    mpz_clear(pq_1);

    mpz_clear(p);
    mpz_clear(q);
    mpz_clear(e);
}

void RSA::decrypt(char* msg) const
{
    mpz_t c, m;
    mpz_init2(c, 1024);
    mpz_init2(m, 1024);

    mpz_import(c, 128, 1, 1, 0, 0, msg);

    // m = c^d mod n
    mpz_powm(m, c, d, n);

    size_t count = (mpz_sizeinbase(m, 2) + 7) / 8;
    memset(msg, 0, 128 - count);
    mpz_export(msg + (128 - count), nullptr, 1, 1, 0, 0, m);

    mpz_clear(c);
    mpz_clear(m);
}

With this:

#include "otpch.h"

#include "rsa.h"

#include <cryptopp/base64.h>
#include <cryptopp/osrng.h>

#include <fstream>
#include <sstream>

static CryptoPP::AutoSeededRandomPool prng;

void RSA::decrypt(char* msg) const
{
    try {
        CryptoPP::Integer m{reinterpret_cast<uint8_t*>(msg), 128};
        auto c = pk.CalculateInverse(prng, m);
        c.Encode(reinterpret_cast<uint8_t*>(msg), 128);
    } catch (const CryptoPP::Exception& e) {
        std::cout << e.what() << '\n';
    }
}

static const std::string header = "-----BEGIN RSA PRIVATE KEY-----";
static const std::string footer = "-----END RSA PRIVATE KEY-----";

void RSA::loadPEM(const std::string& filename)
{
    std::ifstream file{filename};

    if (!file.is_open()) {
        throw std::runtime_error("Missing file " + filename + ".");
     }

    std::ostringstream oss;
    for (std::string line; std::getline(file, line); oss << line);
    std::string key = oss.str();

    if (key.substr(0, header.size()) != header) {
        throw std::runtime_error("Missing RSA private key header.");
    }

    if (key.substr(key.size() - footer.size(), footer.size()) != footer) {
        throw std::runtime_error("Missing RSA private key footer.");
    }

    key = key.substr(header.size(), key.size() - footer.size());

    CryptoPP::ByteQueue queue;
    CryptoPP::Base64Decoder decoder;
    decoder.Attach(new CryptoPP::Redirector(queue));
    decoder.Put(reinterpret_cast<const uint8_t*>(key.c_str()), key.size());
    decoder.MessageEnd();

    try {
        pk.BERDecodePrivateKey(queue, false, queue.MaxRetrievable());

        if (!pk.Validate(prng, 3)) {
            throw std::runtime_error("RSA private key is not valid.");
        }
    } catch (const CryptoPP::Exception& e) {
        std::cout << e.what() << '\n';
    }
}

Finally, I changed RSA.h to look like this:

#ifndef FS_RSA_H_C4E277DA8E884B578DDBF0566F504E91
#define FS_RSA_H_C4E277DA8E884B578DDBF0566F504E91

#include <cryptopp/rsa.h>

#include <string>

class RSA
{
    public:
        RSA() = default;

        // non-copyable
        RSA(const RSA&) = delete;
        RSA& operator=(const RSA&) = delete;

        void loadPEM(const std::string& filename);
        void decrypt(char* msg) const;

    private:
        CryptoPP::RSA::PrivateKey pk;
};

#endif

I generated a new RSA set of keys from Gesior's tool (OTS RSA Generator (https://ots.me/rsa/)), pasted the N value into the OTClient file, and added the key.pem file to the root of the server folder where it belongs. I recompiled with no errors:

admin@OTServer:~/OTServer/build$ make
[  1%] Building CXX object CMakeFiles/tfs.dir/src/networkmessage.cpp.o
[  2%] Building CXX object CMakeFiles/tfs.dir/src/otserv.cpp.o
[  3%] Building CXX object CMakeFiles/tfs.dir/src/protocol.cpp.o
[  5%] Building CXX object CMakeFiles/tfs.dir/src/protocollogin.cpp.o
[  6%] Building CXX object CMakeFiles/tfs.dir/src/rsa.cpp.o
[  7%] Linking CXX executable tfs
[100%] Built target tfs

The server started up, but I could not login; got the "Server was disconnected: Error Code 2" or whatever the message is when the RSA key is wrong. Can anyone help me figure out what I've missed or where I've gone wrong?

Thanks!

 

[fusion32]

See if you get the same problem with the default OT RSA key. It may be that the OTClient isn't setting the key properly?

Also, it is always a good idea to generate your keys locally. Here are some commands if you have OpenSSL:

# Generate RSA1024 key pair
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:1024 -pkeyopt rsa_keygen_pubexp:65537 -outform PEM -out key.pem

# Check key pair
openssl pkey -in key.pem -noout -check

# Export public key
openssl pkey -in key.pem -out pubkey.pem -pubout

# Print PEM contents
openssl pkey -in key.pem -noout -text
openssl pkey -in pubkey.pem -noout -pubin -text

 

[X X X]

I will test with the standard OTLand RSA and see if that works.

Also, it is always a good idea to generate your keys locally. Here are some commands if you have OpenSSL:

For sure, I would generate my keys locally for release. Grabbed them from the generator for making testing easier.

I have a question; some super old versions of TFS had an option in otserv.cpp to manually set p, q and d. Later versions (like mine) you can only set p and q, and it looks like d is calculated inside of rsa.cpp based on p and q, am I understanding that right?

here, specifically:

 // d = e^-1 mod (p - 1)(q - 1)
    mpz_invert(d, e, pq_1);

Main reason I'm doing this in the first place is updating p and q in otserv.cpp and then updating N in OTClient doesn't seem to work. According to this thread here (Client rsa help (https://otland.net/threads/client-rsa-help.252642/)) it's because I have the version that doesn't let you manually set "d". Is that true? It seems weird that TFS would become less customizable...

 

[Iryont]

I have a question; some super old versions of TFS had an option in otserv.cpp to manually set p, q and d. Later versions (like mine) you can only set p and q, and it looks like d is calculated inside of rsa.cpp based on p and q, am I understanding that right?

I was the one to rework that code, so let me answer it. There is no need to manually set d as it can be calculated from p and q. These are basics of RSA algorithm and mathematic behind it. The same goes to n. Everything can be obtained just from p, q and e.

That being said, there is no need to manually set anything except for p, q and e. If something doesn’t work it means you are doing something wrong. The algorithm works just fine.

 

[X X X]

Thanks for answering Iryont! Ok that's good to hear because that's what I understood from reading about RSA. Any ideas what I'm doing wrong then? Here are my steps exactly:
1. I generate a new p and q and add it to otserv.cpp
2. I recompile the source code.
3. I add the new n to the OTClient const.lua file, make sure its saved.
4. Try and login, failed.
I must be generating my values wrong somehow?

EDIT: Everything is working, for some reason my target build tfs was going to a different folder. Regardless, thanks for the replies from both of you and for Iryont for clearing that up!

 

[Iryont]

EDIT: Everything is working, for some reason my target build tfs was going to a different folder. Regardless, thanks for the replies from both of you and for Iryont for clearing that up!

That's good to hear, after all everything works as expected