Environment Setup

The Sidecar is designed to run as a container co-resident to the exchange application it’s serving. However, this is not required; many users of the Sidecar appliance run a discrete cluster (often using Kubernetes) that exchange servers can reach over the network.

Runtime Flag Overview

You can begin by running docker run liveramp/idl-mapper:v2.1.0-stable -help to see available options.

Runtime Flags

Usage of /gobin/idl-mapperd:
  -certificate string
    	Path to file with client certificate and certificate key to use for authentication
  -dry-run
    	Switch for connecting to Check-in service, if 'true' dummy keys will be used
  -help
    	Print this help text and exit
  -id-source string
    	ID source to be set as default (default "liveramp.com")
  -listen string
    	Port or unix socket to listen to (default "unix:///mnt/sock/idl-mapper-socket")
  -separate-passthrough
    	Flag to put Passthrough mappings under a separate source domain
  -shutdown-interval int
    	Number of hours after which mapper will shutdown if it is unauthorized. If 0, shutdown option is disabled. (default 24)

🚧

IPC_LOCK is Required

Note that the Docker commands below include the --cap-add IPC_LOCK flag. Enabling this non-default feature is required to protect the sensitive encryption keys used in the Sidecar appliance.

Client Certificate

The client certificate and client key will be provided by LiveRamp. Running in development mode (by passing the dry-run argument) will skip certificate validation, but will still check for the existence of the flag as a runtime argument.

To obtain a production certificate, you will need to provide LiveRamp with a PGP/GPG public key for secure transmission of your certificate. If you do not already have a GPG key, follow instructions here, though you can skip any Github-specific steps. You will need to provide LiveRamp with the public key, obtained through gpg --armor --export <KEY_ID> or similar.

In the following example, a directory at ~Documents/certs is mounted in the container at /mnt/certs.

dev image example

docker run -v ~/Documents/certs:/mnt/certs --cap-add IPC_LOCK -p 127.0.0.1:5000:5000 liveramp/idl-mapper:latest -listen tcp://:5000 -certificate " " -dry-run true

Connecting to the Sidecar via TCP or Socket

The Sidecar application exposes either a socket or a network interface using TCP to process API requests. While accessing the Sidecar over a Unix socket has a small performance improvement over the TCP connection when the Sidecar is co-resident to the exchange application, we generally recommend using TCP for its relative ease-of-use. In production, the Sidecar is run with a -listen flag, that accepts addresses of the form unix:///path/to/socket or tcp://:port-number.

TCP Network Interface Example

docker run -v ~/Documents/certs:/mnt/certs --cap-add IPC_LOCK -p 127.0.0.1:5000:5000 liveramp/idl-mapper:latest -listen tcp://:5000 -certificate "/mnt/certs/CERTIFICATE_NAME.pem"

Note that the unix socket path should always fall under the /mnt/sock directory on the container, which can be shared with other containers or mounted externally via a docker volume mount at runtime.

Unix Socket Example

docker run -v ~/Documents/certs:/mnt/certs --cap-add IPC_LOCK -v /path/to/host/socket/dir:/mnt/sock liveramp/idl-mapper:idl-mapper:latest -listen unix:///mnt/sock/idl-mapper-socket -certificate "/mnt/certs/CERTIFICATE_NAME.pem"

In the case above, /path/to/host/socket/dir/idl-mapper-socket will appear on the host.

📘

A note on socket sharing

Docker currently doesn’t support sharing sockets over a hypervisor. If you’re testing the Sidecar on a non-Linux machine and try to mount the Sidecar’s socket to your host, the socket itself won’t appear on the host without an error. For the time being, this testing needs to be done either on a VM or in a native Linux environment.

IDSource

Beginning with the Sidecar v2 release, envelopes may contain more than one source of identifiers. In the initial v2 release, identifiers from LiveRamp (IDSource of liveramp.com) as well as The Trade Desk UID2 ids (IDSource of uidapi.com) are included. Specifying the IDSource as a runtime flag will select the id used in the single-id /map endpoint; by default liveramp.com IDs are selected, but this can be set to a different default by passing the appropriate IDSource value. See API endpoints for more information about working with multiple IDSources, including multiple IDs in a single /map request.

Separate Passthrough

You may wish to send envelopes to downstream platforms in the event that they run their own sidecar instances. In this case, you can either skip sidecar processing entirely, or request that LiveRamp map seat(s) as "passthrough".

By default, passthrough seats will be organized under the liveramp.com IDSource:

JSON

[
  {
    "source": "liveramp.com",
    "mapping": {
      "Envelope Passthrough Seat": "Apgt2tWKDEAsfNlOn8_DTpgojfVeGGhr0mbXul_rx4kAj8_Tkxfk3U3Rwdk0fEQ09c16KuWr3F0kwxvyoIL0tLVPfKZO3kVHfFfWtoaDWVz81r2-cEVSn5Hc21eUPyHZNijGW7HdID6gSXozFVwG2PIDy1DYs7xzpdowOTZol8cNfCw512vlo0Cn9ibPIJ0j1Jdf5oU8rszMxKsMHx8a6Bp1NB1rV7B2s_qx9tNlxjxP7V1OqMI9hJnRIisbOx42-paBMrZmMtMs7YAjVxPlKqEFZBmGlAdeqpEQlwNfpCxcwJ5zCjsHjK9IAB9ga7zszBMvh6VfcaRfmEpnZTmbgCZqAKkKNbR6UzDP8yp2bB1JGhN_MmhAhE0AnIHclBhdqK6a5AHqC-TdEz-pMsUNBDPhFXkchoGcRvm_E_Xef2Ydtu88Xy0byRAOtT1hsCAcujM24RtFrbzX9T4",
      "Test Seat 1": "XY9999Ky296WYD8-BuzIlZabukhPeVGr9s5V80Q4b_LRYWu0g"
    }
  },
  {
    "source": "uidapi.com",
    "mapping": {
      "Test Seat 1": "AgAAABUBOEOjShB4rXDrTQT27nejDDTCuSGcXBt6DbdE+HwxMjOLF5BcYbl02PBDSgxvzOuuM86EV84WmhUWI7x7wIpLyfkK5ZGMBQCVzvc+lhtKXgN8fvQIXrvI+x/KFwBiMHOMAa6axWECnJ6SRP65Ih0RBtAdrx9xoPkT1/A1QjzXTw==",
    }
  }
]

But by adding -separate-passthrough to your runtime flags, you can treat it as a discrete IDSource:

Passthrough Flag Enabled

[
  ...
  },
  {
    "source": "passthrough",
    "mapping": {
      "Envelope Passthrough Seat": "Apgt2tWKDEAsfNlOn8_DTpgojfVeGGhr0mbXul_rx4kAj8_Tkxfk3U3Rwdk0fEQ09c16KuWr3F0kwxvyoIL0tLVPfKZO3kVHfFfWtoaDWVz81r2"
    }
  }
]

Security Measures

The application runs completely memory-locked within the container, in order to prevent any sensitive keys from being written to disk, so the application must be given the permissions and resources to effectively lock enough memory. The application shouldn’t need more than 10MB, although alongside the image operating system, we recommend allocating ~300MB RAM.

The application will not run unless the container has its core file size hard limit set to 0, in order to prevent any sensitive keys being written out in core dumps. If the container is run with this value set to anything above 0, a warning message will be printed to STDOUT and the value will be overwritten to 0.

❗️

Important

Attempting to modify directories other than /mnt/sock and /mnt/secrets on the Sidecar can result in triggering tamper-prevention mechanisms, which could result in an account-wide disruption of service.

Network Traffic and Telemetry

The Sidecar container needs to be able to communicate with LiveRamp’s services at sidecar.liveramp.com to periodically (every one and five minutes) check in, obtain and renew keys and to send telemetry and crash reports. Note that no personal data is sent to LiveRamp, only aggregated counts and information about the Docker runtime. In most cases, connectivity requires no special Docker setup, however it may be impacted if you’re using a special networking mode.

📘

Sidecar IP Addresses

The Sidecar will connect to sidecar.liveramp.com. If you need to whitelist specific IP addresses, you may use 35.193.49.161 and 35.188.138.165.

See API endpoints for information on what aggregated telemetry is collected.

ARM64 Support

The default Sidecar image is intended for use on x86 systems, but LiveRamp now offers initial support for ARM64 architectures. Images have a suffix of -arm64.