Skip to main content

Compiler and Runtime

In order to use Protobuf format, we need three things:

  1. The source .proto files.
  2. A compiler that compiles the source files.
  3. A runtime that loads the compiled files.

Compiler​

There is a main compiler that can compile .proto files into Python, Go, Rust, C#, Java and C++. Once installed its command is protoc.

For JavaScript, we have two options:

  1. google-protobuf

    • This is the official way.
    • Requires a build of the protoc source, with some modifications so the protoc command will become capable of compiling into JavaScript output.
    • Once installed its command is the same protoc as before but with a new --js-out option available.

  2. protobufjs

    • This is a 3rd-party package.
    • Requires no extra step.
    • Once installed, its command is pbjs.

We will move forward with the option 2, mainly because it's simpler and requires less explanation.

Install Main Compiler​

winget install protobuf
protoc --version # libprotoc 31.1
Manual install
  1. Go to "Releases" page of the compiler's Github repository.
  2. Under "Assets", download the protoc-31.1-win64.zip file.
  3. Extract the zip file into for example the C:\protoc-31.1-win64 directory.
  4. Add the path to the extracted files to your computer's path variable.
    1. Press win + r keys.
    2. Type rundll32.exe sysdm.cpl,EditEnvironmentVariables
    3. Under User Variables, select Path and click Edit button, and in opened form click Add.
    4. Put C:\protoc-31.1-win64\bin, and click OK, and another OK.

Install JavaScript-Specific Compiler​

npm install protobufjs -g
pbjs

Runtime​

To load and use the compiled files in our code, we need to install the Protobuf runtime in our programming language environment, which is in the form of a library or package.

npm i protobufjs
pbjs # to check version

# protobuf.js v1.1.3 CLI for JavaScript
# ...

Compiler and Runtime Versions​

When working with Protobuf, there is a concept called Cross-Version Runtime. It simply means you cannot just use any version of the compiler with any version of the runtime willغ-nilly. The output of the compiler is called "Generated Code" or GenCode for short. The runtime has to be able to load this GenCode.

The compiler and the runtime have their own versions. You can think of the compiler as the one who writes some things, and the runtime as the one that has to read what's been written. Now imagine the writer in a newer version, writes in a way that only the reader of the same version can understand it.

Here is the officiall recommandation of the Protobuf itself. Altough you can get away with a new runtime and old GenCode, The simplest approach is to match both of their versions to a specific publish date. The below Node.js scripts shows how to find proper compiler version for a runtime.

// `protoc`

url =
  'https://api.github.com/repos/protocolbuffers/protobuf/releases?per_page=100&page=1';
r = await (await fetch(url)).json();
a = r.map((i) => [i.tag_name, i.published_at]);

// `protobuf` pypi pkg
url = 'https://pypi.org/pypi/protobuf/json';
r = await (await fetch(url)).json();
b = Object.keys(r.releases).map((k) => [k, r.releases[k][0].upload_time]);

// find out proper compiler version for `protobuf` pypi pkg 3.20.1 (for example)
b.find((i) => i[0] === '3.20.1'); // ['3.20.1', '2022-04-22T02:03:01']   py pkg ver
a.find((i) => i[1].includes('2022-04')); // ['v3.20.1', '2022-04-22T02:27:33Z'] proper compiler ver

How to Check If Your Runtime is Ok with GenCode​

In the Python case, you should compile your .proto files, and then try to import them in a script (that's it, just import them and nothing else), then try to run the script. If there exists a version mismatch, you will see proper warnings that say as such. If you don't see any warning, that means you have done everything correctly.