TFS 1.7 Server: RSA PEM Key Issue on Modern Linux / OpenSSL 3.x

[Ausmax]

Hi everyone,


I’m working on releasing a public TFS 1.7 server and ran into a problem with the RSA key (tfs.pem) on modern Linux/WSL:

• I generated a PEM key using OpenSSL 3.x (Ubuntu 22.04), but TFS 1.7 cannot read it, showing:

ERROR: Error while reading PEM data: error:00000000:lib(0)::reason(0)

• I tried openssl genrsa, openssl genpkey, and converting line endings — everything produces a valid key according to openssl rsa -check, but TFS still refuses it.

My environment:

• TFS 1.7 compiled on Linux x64
• OpenSSL 3.x (Ubuntu 22.04 / WSL)
• Lua 5.2.4
• I want to release this server publicly, so the key must be unique for security

What I’m looking for:

1. A reliable way to generate a TFS 1.7–compatible PEM on Linux or Windows.
2. Advice from anyone who successfully released a public server with TFS 1.7 on modern systems.
3. Any tips for community-safe deployment, keeping the PEM secure.

I’d appreciate any guidance, scripts, or recommended workflows for generating a working RSA key for TFS 1.7.


Thanks in advance!

Post automatically merged: Feb 7, 2026

or is there any updated Guides the guides Otland has are over 4 years old and following the guide i just get Error after error can't install this can;t locate that

 

[fusion32]

Does it work with the default key? What's the command used to generate the key? Running something like the command below usually works for me, unless there is some issue with the key loader.

openssl genrsa -f4 -out key.pem 1024

 

[Ausmax]

Thanks for the reply.





Yes, we tried the default openssl genrsa -f4 -out key.pem 1024 command, but on modern systems (Ubuntu 22.04 / OpenSSL 3.x) it does not generate a classic PKCS#1 key anymore. It outputs:

-----BEGIN PRIVATE KEY-----

instead of the format TFS 1.7 expects:

-----BEGIN RSA PRIVATE KEY-----

Even though openssl rsa -check reports the key as valid, TFS 1.7 fails to parse PKCS#8 keys, which results in the generic:

ERROR: Error while reading PEM data

This happens with:

• genrsa
• genpkey
• converted keys
  as long as OpenSSL 3.x defaults are used.

The issue is not the command itself, but OpenSSL 3.x dropping legacy RSA output unless explicitly forced.





What does work reliably:

• Using openssl genrsa -traditional -f4 -out tfs.pem 1024 with the legacy provider enabled, or
• Generating the key on an older OpenSSL (1.1.1, e.g. Ubuntu 20.04 or Docker), which produces a PKCS#1 key by default.

Regarding the “default key”: I’m intentionally not using it since this is meant to be a public server, and reusing a known RSA key is a security risk.





So at this point the blocker isn’t Boost or Lua (those are resolved), it’s purely OpenSSL 3.x vs TFS 1.7 compatibility.





If you or anyone has successfully released a public TFS 1.7 server on OpenSSL 3.x without forcing legacy RSA output, I’d be very interested to hear how.





Thanks again for the input.

 

[fusion32]

What does work reliably:

• Using openssl genrsa -traditional -f4 -out tfs.pem 1024 with the legacy provider enabled, or
• Generating the key on an older OpenSSL (1.1.1, e.g. Ubuntu 20.04 or Docker), which produces a PKCS#1 key by default.

Regarding the “default key”: I’m intentionally not using it since this is meant to be a public server, and reusing a known RSA key is a security risk.

If you or anyone has successfully released a public TFS 1.7 server on OpenSSL 3.x without forcing legacy RSA output, I’d be very interested to hear how.

Ah yes, I forgot that -traditional bit. But if it works with that, then it should be solved? There shouldn't be too much of a difference between PKCS#1 and the later formats, except that the later formats support other types of keys (I think?) while PKCS#1 was RSA only (I think?). Anyway, I was asking about the default key because it seems to use the regular "BEGIN PRIVATE KEY" headers, at least on the latest commit.