Remote Publishing#

Advanced

Package Manager supports adding packages to local Python or R repositories remotely. This feature allows easier integration with separate CI/CD package build pipelines or adding packages directly from a development environment.

Remote Token Authentication#

Local sources support remote token authentication. You can use remote authentication to upload packages via the rspm add command in the CLI or, additionally, Python packages also support twine.

An API token needs to be generated to allow a user to upload packages to the source. To create the token, run the command on the server:

$ rspm create token --description="Local Python token" --sources=local-python-src --expires=30d --scope=sources:write
<< Generated an access token. Be sure to record this token immediately since you will not be able to retrieve it later.
<< eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJwYWNrYWdlbWFuYWdlciIsImV4cCI6MTY1ODU4MjA3OCwianRpIjoiYmM5ZTg1NGYtNGNlNy00Zjc4LTlhMmMtZDliYzRlYTQ0NGVkIiwiaWF0IjoxNjU1OTkwMDc4LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjQyNDIvIiwic2NvcGVzIjp7InNvdXJjZXMiOiIzYTI4NjFhYi0xNWYwLTRjM2MtODZlMy0xNjNkMTY0ZDE0ZDYifX0.BWJXLworo44Nvfrh5a2hm_NIqgUoXTLjQlxyy7uaSWk

Uploading Packages Remotely with the CLI#

On the remote machine, export the correct environment variables and download the CLI:

$ export PACKAGEMANAGER_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJwYWNrYWdlbWFuYWdlciIsImV4cCI6MTY1ODU4MjA3OCwianRpIjoiYmM5ZTg1NGYtNGNlNy00Zjc4LTlhMmMtZDliYzRlYTQ0NGVkIiwiaWF0IjoxNjU1OTkwMDc4LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjQyNDIvIiwic2NvcGVzIjp7InNvdXJjZXMiOiIzYTI4NjFhYi0xNWYwLTRjM2MtODZlMy0xNjNkMTY0ZDE0ZDYifX0.BWJXLworo44Nvfrh5a2hm_NIqgUoXTLjQlxyy7uaSWk
$ export PACKAGEMANAGER_ADDRESS=<https://my-rspm-server.org:4443>

# Download the CLI
$ curl -fLOJH "Authorization: Bearer ${PACKAGEMANAGER_TOKEN}" "${PACKAGEMANAGER_ADDRESS}/__api__/download"
$ chmod +x rspm

$ export PACKAGEMANAGER_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJwYWNrYWdlbWFuYWdlciIsImV4cCI6MTY1ODU4MjA3OCwianRpIjoiYmM5ZTg1NGYtNGNlNy00Zjc4LTlhMmMtZDliYzRlYTQ0NGVkIiwiaWF0IjoxNjU1OTkwMDc4LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjQyNDIvIiwic2NvcGVzIjp7InNvdXJjZXMiOiIzYTI4NjFhYi0xNWYwLTRjM2MtODZlMy0xNjNkMTY0ZDE0ZDYifX0.BWJXLworo44Nvfrh5a2hm_NIqgUoXTLjQlxyy7uaSWk
$ export PACKAGEMANAGER_ADDRESS=<https://my-rspm-server.org:4443>

# Download the CLI
$ curl -fLOJH "Authorization: Bearer ${PACKAGEMANAGER_TOKEN}" "${PACKAGEMANAGER_ADDRESS}/__api__/download?os=darwin"
$ chmod +x rspm

$Env:PACKAGEMANAGER_TOKEN = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJwYWNrYWdlbWFuYWdlciIsImV4cCI6MTY1ODU4MjA3OCwianRpIjoiYmM5ZTg1NGYtNGNlNy00Zjc4LTlhMmMtZDliYzRlYTQ0NGVkIiwiaWF0IjoxNjU1OTkwMDc4LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjQyNDIvIiwic2NvcGVzIjp7InNvdXJjZXMiOiIzYTI4NjFhYi0xNWYwLTRjM2MtODZlMy0xNjNkMTY0ZDE0ZDYifX0.BWJXLworo44Nvfrh5a2hm_NIqgUoXTLjQlxyy7uaSWk"
$Env:PACKAGEMANAGER_ADDRESS = "https://my-rspm-server.org:4443"

# Download the CLI
$urlCLI = "$Env:PACKAGEMANAGER_ADDRESS/__api__/download?os=windows"
$outputCLI = "rspm.exe"
$wc = New-Object System.Net.WebClient
$wc.Headers['Authorization'] = "Bearer $Env:PACKAGEMANAGER_TOKEN"
$wc.DownloadFile($urlCLI, $outputCLI)

# Alternatively, on Windows 10 (v1803+), you can download the CLI with curl
curl -OutFile rspm.exe -Headers @{ Authorization = "Bearer $Env:PACKAGEMANAGER_TOKEN"} "$Env:PACKAGEMANAGER_ADDRESS/__api__/download?os=windows"

After the environment variables are set, the remote machine can use CLI commands that support remote use:

$ ./rspm add --source=local-python-src --path=/path/to/packages

.\rspm.exe add --source=local-python-src --path=C:\path\to\packages

Uploading Python Packages Remotely with Twine#

Twine is a tool that can be used to upload Python packages that have been built locally. To learn more about its full feature set, review the Twine documentation.

To start, install Twine on the system that will be uploading packages:

$ pip install twine

Twine Configuration#

There are several ways that Twine can be used to remotely authenticate with Package Manager:

• Using the TWINE_* environment variables.
• Configuring a .pypirc file.
• Authenticating with keyring.

Using the TWINE_* environment variables#

The three environment variables Twine uses are:

• TWINE_REPOSITORY_URL
• TWINE_USERNAME
• TWINE_PASSWORD

The TWINE_REPOSITORY_URL environment variable is the address that is used to upload to Package Manager remotely, ending with the endpoint /upload/pypi/{source-name}. Since we are using token authentication, TWINE_USERNAME will always be set to __token__. Finally, TWINE_PASSWORD is set to the token that was generated above. To put this all together, the following is an example of what the variables could be set to:

$ export TWINE_REPOSITORY_URL=https://packagemanager.posit.co/upload/pypi/local-python-src
$ export TWINE_USERNAME=__token__
$ export TWINE_PASSWORD=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJwYWNrYWdlbWFuYWdlciIsImV4cCI6MTY1ODU4MjA3OCwianRpIjoiYmM5ZTg1NGYtNGNlNy00Zjc4LTlhMmMtZDliYzRlYTQ0NGVkIiwiaWF0IjoxNjU1OTkwMDc4LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjQyNDIvIiwic2NvcGVzIjp7InNvdXJjZXMiOiIzYTI4NjFhYi0xNWYwLTRjM2MtODZlMy0xNjNkMTY0ZDE0ZDYifX0.BWJXLworo44Nvfrh5a2hm_NIqgUoXTLjQlxyy7uaSWk

With the environment variables set, you can easily upload with Twine:

$ twine upload dist/*

Configuring a .pypirc file#

An alternative approach to using the Twine environment variables is to configure a .pypirc file. Twine looks for this file in the ~/.pypirc location by default. If instead a user wants to pass in a .pypirc file that is not in that location, they can use Twine's --config-file flag to point to that file in a different directory.

The .pypirc looks like the following:

[distutils]
index-servers =
   package-manager

[package-manager]
repository = https://packagemanager.posit.co/upload/pypi/local-python-src
username = __token__
password = eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJwYWNrYWdlbWFuYWdlciIsImV4cCI6MTY1ODU4MjA3OCwianRpIjoiYmM5ZTg1NGYtNGNlNy00Zjc4LTlhMmMtZDliYzRlYTQ0NGVkIiwiaWF0IjoxNjU1OTkwMDc4LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjQyNDIvIiwic2NvcGVzIjp7InNvdXJjZXMiOiIzYTI4NjFhYi0xNWYwLTRjM2MtODZlMy0xNjNkMTY0ZDE0ZDYifX0.BWJXLworo44Nvfrh5a2hm_NIqgUoXTLjQlxyy7uaSWk

Then to use this file to upload with Twine:

$ twine upload -r package-manager dist/*

The .pypirc file makes it easy to upload to various sources quickly. When you create another repository definition in the .pypirc file, it is made available to use with Twine's -r flag.

Authenticating with keyring#

An alternative approach to putting the username/password directly into a .pypirc file or environment variables is to store them securely in a keyring. Keyring is automatically installed with Twine, so it is available to use out-of-the-box.

If a user is using the .pypirc method, their file would now look like:

[distutils]
index-servers =
   package-manager

[package-manager]
repository = https://packagemanager.posit.co/upload/pypi/local-python-src

If a user is instead using the Twine environment variables, now all they need to set is:

$ export TWINE_REPOSITORY_URL=https://packagemanager.posit.co/upload/pypi/local-python-src

Then, to store the username and password for the repository, a user can save them with keyring:

$ keyring set https://packagemanager.posit.co/upload/pypi/local-python-src __token__
<< Password for '__token__' in 'https://packagemanager.posit.co/upload/pypi/local-python-src':

The password has been saved securely within the keyring and can be used by Twine. When uploading with Twine, it will now prompt for the username and check keyring to see if it has a password. If it does, it will successfully upload the distributions:

$ cd directory/of/internal/example_package
$ twine upload -r package-manager dist/*
<< Enter your username: __token__
<< Uploading distributions to https://packagemanager.posit.co/upload/pypi/local-python-src
<< Uploading example_package.whl
<< 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.1/8.1 MB • 00:02 • 3.5 MB/s

Twine Package Signing#

Twine and Package Manager also support uploading signed packages. Package Manager detects when a signed file is uploaded and stores the file with the Python package. A user can see this by navigating to the package in the UI and clicking on the specific distribution. If the distribution has a green SIGNED label, then it successfully uploaded the .asc file with the Python distribution. To download this file for future use, a user can append .asc to the URL for that specific distribution to download the signed file.

There are two methods for uploading signed files:

1. Manually uploading pre-signed files.
2. Automatically signing with Twine.

Manually uploading pre-signed files#

If you already have a package pre-signed, the Twine command is:

$ twine upload example_package.whl example_package.whl.asc

Automatically signing with Twine#

There is also the --sign flag that instructs Twine to sign the files before uploading:

$ twine upload --sign example_package.whl
<< Uploading distributions to https://packagemanager.posit.co/upload/pypi/local-python-src
<< Signing example_package.whl
<< Uploading example_package.whl
<< 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.1/8.1 MB • 00:02 • 3.5 MB/s