Set up Trusted Execution Environment (TEE)
In case the ParaTime you want to run does not require the use of a TEE (e.g. Intel SGX), you can skip setting up a TEE.
If the ParaTime is configured to run in a TEE (currently only Intel SGX), you must make sure that your system supports running SGX enclaves. This requires that your hardware has SGX support, that SGX support is enabled and that the additional driver and software components are properly installed and running.

Install SGX Linux Driver

Oasis Core currently only supports the legacy (out-of-tree) Intel SGX Linux driver.
Support for the new Intel SGX support in mainline Linux kernels since version 5.11 is being tracked in oasis-core#3651.

Ubuntu 18.04/16.04

A convenient way to install the SGX Linux driver on Ubuntu 18.04/16.04 systems is to use the Fortanix's APT repository and its DKMS package.
First add Fortanix's APT repository to your system:
1
echo "deb https://download.fortanix.com/linux/apt xenial main" | sudo tee /etc/apt/sources.list.d/fortanix.list >/dev/null
2
curl -sSL "https://download.fortanix.com/linux/apt/fortanix.gpg" | sudo -E apt-key add -
Copied!
And then install the intel-sgx-dkms package:
1
sudo apt update
2
sudo apt install intel-sgx-dkms
Copied!
To determine that, run dmesg | grep -i sgx and observe if a line like the following is shown:
1
[ 4.991649] sgx: intel_sgx: Intel SGX DCAP Driver v1.33
Copied!
If that is the case, you need to blacklist the Intel SGX DCAP driver's module by running:
1
echo "blacklist intel_sgx" | sudo tee -a /etc/modprobe.d/blacklist-intel_sgx.conf >/dev/null
Copied!

Fedora 34/33

A convenient way to install the SGX Linux driver on Fedora 34/33 systems is to use the Oasis-provided Fedora Package for the Legacy Intel SGX Linux Driver.

Other Distributions

Go to Intel SGX Downloads page and find the latest "Intel SGX Linux Release" (not "Intel SGX DCAP Release") and download the "Intel (R) SGX Installers" for your distribution. The package will have driver in the name (e.g., sgx_linux_x64_driver_2.11.0_2d2b795.bin).

Verification

After installing the driver and restarting your system, make sure that the /dev/isgx device exists.

Ensure /dev is NOT Mounted with the noexec Option

Newer Linux distributions usually mount /dev with the noexec mount option. If that is the case, it will prevent the enclave loader from mapping executable pages.
Ensure your /dev (i.e. devtmpfs) is not mounted with the noexec option. To check that, use:
1
cat /proc/mounts | grep devtmpfs
Copied!
To temporarily remove the noexec mount option for /dev, run:
1
sudo mount -o remount,exec /dev
Copied!
To permanently remove the noexec mount option for /dev, add the following to the system's /etc/fstab file:
1
devtmpfs /dev devtmpfs defaults,exec 0 0
Copied!
This is the recommended way to modify mount options for virtual (i.e. API) file system as described in systemd's API File Systems documentation.

Install AESM Service

To allow execution of SGX enclaves, several Architectural Enclaves (AE) are involved (i.e. Launch Enclave, Provisioning Enclave, Provisioning Certificate Enclave, Quoting Enclave, Platform Services Enclaves).
Communication between application-spawned SGX enclaves and Intel-provided Architectural Enclaves is through Application Enclave Service Manager (AESM). AESM runs as a daemon and provides a socket through which applications can facilitate various SGX services such as launch approval, remote attestation quote signing, etc.

Ubuntu 20.04/18.04/16.04

A convenient way to install the AESM service on Ubuntu 20.04/18.04/16.04 systems is to use the Intel's official Intel SGX APT repository.
First add Intel SGX APT repository to your system:
1
echo "deb https://download.01.org/intel-sgx/sgx_repo/ubuntu $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/intel-sgx.list >/dev/null
2
curl -sSL "https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key" | sudo -E apt-key add -
Copied!
And then install the sgx-aesm-service, libsgx-aesm-launch-plugin and libsgx-aesm-epid-plugin packages:
1
sudo apt update
2
sudo apt install sgx-aesm-service libsgx-aesm-launch-plugin libsgx-aesm-epid-plugin
Copied!
The AESM service should be up and running. To confirm that, use:
1
sudo systemctl status aesmd.service
Copied!

Docker-enabled System

An easy way to install and run the AESM service on a Docker-enabled system is to use Fortanix's AESM container image.
Executing the following command should (always) pull the latest version of Fortanix's AESM Docker container, map the /dev/isgx device and /var/run/aesmd directory and ensure AESM is running in the background (also automatically started on boot):
1
docker run \
2
--pull always \
3
--detach \
4
--restart always \
5
--device /dev/isgx \
6
--volume /var/run/aesmd:/var/run/aesmd \
7
--name aesmd \
8
fortanix/aesmd
Copied!

Podman-enabled System

Similarly to Docker-enabled systems, an easy way to install and run the AESM service on a Podman-enabled system is to use Fortanix's AESM container image.
First, create the container with:
1
sudo podman create \
2
--pull always \
3
--device /dev/isgx \
4
--volume /var/run/aesmd:/var/run/aesmd:Z \
5
--name aesmd \
6
docker.io/fortanix/aesmd
Copied!
Then generate the container-aesmd.service systemd unit file for it with:
1
sudo podman generate systemd --restart-policy=always --time 10 --name aesmd | \
2
sed "/\[Service\]/a RuntimeDirectory=aesmd" | \
3
sudo tee /etc/systemd/system/container-aesmd.service
Copied!
Finally, enable and start the container-aesmd.service with:
1
sudo systemctl enable container-aesmd.service
2
sudo systemctl start container-aesmd.service
Copied!
The AESM service should be up and running. To confirm that, use:
1
sudo systemctl status container-aesmd.service
Copied!
To see the logs of the AESM service, use:
1
sudo podman logs -t -f aesmd
Copied!

Check SGX Setup

In order to make sure that your SGX setup is working, you can use the sgx-detect tool from the sgxs-tools Rust package.
There are no pre-built packages for it, so you will need to compile it yourself.
sgxs-tools must be compiled with a nightly version of the Rust toolchain since they use the #![feature] macro.

Install Dependencies

Make sure you have the following installed on your system:
On Fedora, you can install all the above with:
1
sudo dnf install gcc protobuf-compiler pkg-config openssl-devel
Copied!
On Ubuntu, you can install all the above with:
1
sudo apt install gcc protobuf-compiler pkg-config libssl-dev
Copied!

Install Rust Nightly

We follow Rust upstream's recommendation on using rustup to install and manage Rust versions.
rustup cannot be installed alongside a distribution packaged Rust version. You will need to remove it (if it's present) before you can start using rustup.
Install rustup by running:
1
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Copied!
If you want to avoid directly executing a shell script fetched the internet, you can also download rustup-init executable for your platform and run it manually. This will run rustup-init which will download and install the latest stable version of Rust on your system.
Install Rust nightly with:
1
rustup install nightly
Copied!

Build and Install sgxs-tools

1
cargo +nightly install sgxs-tools
Copied!

Run sgx-detect Tool

After the installation completes, run sgx-detect to make sure that everything is set up correctly.
When everything works, you should get output similar to the following (some things depend on hardware features so your output may differ):
1
Detecting SGX, this may take a minute...
2
✔ SGX instruction set
3
✔ CPU support
4
✔ CPU configuration
5
✔ Enclave attributes
6
✔ Enclave Page Cache
7
SGX features
8
✔ SGX2 ✔ EXINFO ✔ ENCLV ✔ OVERSUB ✔ KSS
9
Total EPC size: 92.8MiB
10
✘ Flexible launch control
11
✔ CPU support
12
? CPU configuration
13
✘ Able to launch production mode enclave
14
✔ SGX system software
15
✔ SGX kernel device (/dev/isgx)
16
✘ libsgx_enclave_common
17
✔ AESM service
18
✔ Able to launch enclaves
19
✔ Debug mode
20
✘ Production mode
21
✔ Production mode (Intel whitelisted)
Copied!
The important part is the checkbox under Able to launch enclaves in both Debug mode and Production mode (Intel whitelisted).
In case you encounter errors, see the list of common SGX installation issues for help.

Troubleshooting

See the general troubleshooting section, before proceeding with ParaTime node-specific troubleshooting.

Missing libsgx-aesm-epid-plugin

If you are encountering the following error message in your node's logs:
1
failed to initialize TEE: error while getting quote info from AESMD: aesm: error 30
Copied!
Ensure you have all required SGX driver libraries installed as listed in Install SGX Linux Driver section. Previous versions of this guide were missing the libsgx-aesm-epid-plugin.

Unable to Launch Enclaves

If running sgx-detect --verbose reports:
1
🕮 SGX system software > Able to launch enclaves > Debug mode
2
The enclave could not be launched.
3
4
debug: failed to load report enclave
5
debug: cause: failed to load report enclave
6
debug: cause: Failed to map enclave into memory.
7
debug: cause: Operation not permitted (os error 1)
Copied!
Last modified 1mo ago