Quick Start#
This guide is to help administrators configure Posit Package Manager for your organization's specific use cases.
Prerequisites#
-
If you haven't already, follow the instructions for installing and licensing Package Manager.
-
Ensure your user is a member of the
rstudio-pm
group:Replace
<USER>
with the desired Package Manager admin user. After running this command, restart your shell. -
Finally, configure access to the
rspm
binary in one of two ways:- Set up an alias:
alias rspm='/opt/rstudio-pm/bin/rspm'
- Add the
rspm
CLI binary to your system path:export PATH=/opt/rstudio-pm/bin:$PATH
- Set up an alias:
Overview#
Setting up Package Manager involves configuring sources and repositories. A source contains packages from a specific location, for example a public repository or local files. A repository then subscribes to one or more sources to expose the packages to clients.
graph BT
cranlike("repository:<br/>/all")
cran(source:<br/>cran)
local(source:<br/>internal R packages)
cranlike -->|subscribe| cran
cranlike --> |subscribe| local
If you have additional security challenges, we recommend viewing the Security and Compliance section on the left-hand side, including topics like Server Security, Auditing, and Package Blocking.
The right sidebar contains quick start guides for configuring various types of sources. Follow the ones that are best suited for your use case(s):
- Provide a full mirror of a public repository
- Provide snapshots of a public repository from specific dates
- Create a curated subset of a public repository
- Share internally-developed packages
- Provide custom binary versions of your packages
Once you have configured the server, share the URL to the web interface with your users. Users can set up R, Python, or RStudio Desktop to use Package Manager by following the instructions included in each repository's Setup page.
Tip
Posit Workbench can be configured to use Package Manager without requiring user setup. For more information, see the configuration instructions.
Serving CRAN Packages#
A common use case for Package Manager is making CRAN packages available in environments with restricted internet access.
To make the CRAN packages available, create a repository and subscribe it to the built-in source named cran
:
# Create a repository:
$ rspm create repo --name=cran --description='Access CRAN packages'
# Subscribe the repository to the cran source:
$ rspm subscribe --repo=cran --source=cran
After completing these steps, the cran
repository is available in the web interface.
CRAN and the Posit Package Service#
The CRAN source retrieves its metadata from the Posit Package Service automatically. Package Manager checks for CRAN updates from the Posit Package Service every 10 minutes. If you need to force an immediate update, you can synchronize manually:
Note
Although Package Manager checks for updates regularly, the Posit Package Service may not update packages every day.
Serving PyPI Packages#
Another common use case for Package Manager is making PyPI (the Python Package Index) packages available in environments with restricted internet access.
To make the PyPI packages available, create a repository and subscribe it to the built-in source named pypi
. Package Manager automatically retrieves PyPI packages and metadata from the Posit Package Service.
# Create a Python repository:
$ rspm create repo --name=pypi --type=python --description='Access PyPI packages'
<< Repository: pypi - Python
# Subscribe the repository to the PyPI source:
$ rspm subscribe --repo=pypi --source=pypi
<< Repository: pypi
<< Sources:
<< --pypi (Python)
When the PyPI source is required by at least one repo, future updates will occur automatically as updates become available.
Tip
Configure additional storage before enabling PyPI packages to handle the large metadata and package downloads.
For more information on Package Manager's PyPI support, see the PyPI Mirror section of the documentation.
Serving Bioconductor Packages#
Package Manager can serve Bioconductor packages from two types of repositories:
- Bioconductor repositories provide multiple versions of Bioconductor side by side and serve as a Bioconductor mirror. Packages are installed from these repositories using the BiocManager package. This is the most common way to make Bioconductor packages available.
- R repositories provide a single version of Bioconductor in a CRAN-like repository. Packages are installed from these repositories using
install.packages
. Unlike Bioconductor repositories, R repositories allow CRAN, Local, and Git packages to be served alongside Bioconductor packages in the same repository.
The appropriate repository type to use depends on how your organization uses Bioconductor.
Bioconductor Repositories#
To create a Bioconductor repository, run the following command:
Note
All Bioconductor versions will be synced and automatically added to the repository.
# Create a Bioconductor repository named 'bioconductor':
$ rspm create repo --type=bioconductor --name=bioconductor
<< Repository: bioconductor - Bioconductor
The Bioconductor source retrieves its metadata from the Posit Package Service automatically. Package Manager checks for Bioconductor updates from the Posit Package Service every 10 minutes. If you need to force an immediate update, you can synchronize manually:
Note
Although Package Manager checks for updates regularly, the Posit Package Service may not update packages every day.
We also recommend creating a repository that provides compatible CRAN snapshots for past Bioconductor releases. To provide CRAN snapshots, create a repository that subscribes only to CRAN:
# Check if an existing repository already subscribes to CRAN only:
$ rspm list
<< ...
<< cran - R
<< - cran (CRAN)
# If not, run the following commands:
# Create a repository named 'cran':
$ rspm create repo --name=cran --description='Access CRAN packages'
# Subscribe the repository to the CRAN source:
$ rspm subscribe --repo=cran --source=cran
The Bioconductor repository should now be available in the web interface and ready to use. If a CRAN repository was created, the Setup page for the Bioconductor repository will provide instructions for configuring an appropriate CRAN snapshot in the Using a CRAN Snapshot section.
R Repositories#
Alternatively, you can serve a single version of Bioconductor in an R repository.
First, list the available Bioconductor versions and decide which version you need:
# List available Bioconductor versions:
$ rspm list bioconductor versions
<< Available Bioconductor versions:
<< 3.20
<< 3.19
<< 3.18
<< ...
Then, create an R repository and subscribe it to the bioconductor
source, specifying the version you need:
# Create a repository for Bioconductor 3.11 named 'bioconductor-3.11':
$ rspm create repo --name=bioconductor-3.11 --description='Access Bioconductor 3.11 packages'
# Subscribe the repository to the Bioconductor source:
$ rspm subscribe --repo=bioconductor-3.11 --source=bioconductor --version=3.11
For past Bioconductor releases, we also recommend serving CRAN packages from this repository that are compatible with the Bioconductor version. To add the appropriate CRAN packages:
- Find the last date that the Bioconductor release was current. This is usually the date of the next Bioconductor release, which can be found in the Bioconductor Release Announcements.
- Find a CRAN snapshot that coincides with this date using the
rspm list cran snapshots
command. - Create a CRAN Snapshot source and add it to the repository.
For example, to add CRAN packages for Bioconductor 3.11 (current until October 28, 2020):
# List available CRAN snapshots:
$ rspm list cran snapshots
<< ...
<< 2020-10-29
<< 2020-10-28
<< 2020-10-27
# Create a CRAN Snapshot source for October 28, 2020:
$ rspm create source --type=cran-snapshot --snapshot=2020-10-28 --name=cran-2020-10-28
# Subscribe the CRAN Snapshot source:
$ rspm subscribe --repo=bioconductor-3.11 --source=cran-2020-10-28
<< Repository: bioconductor-3.11
<< Sources:
<< - bioconductor (Bioconductor - 3.11)
<< - cran-2020-10-28 (CRAN Snapshot - 2020-10-28)
Serving CRAN Snapshots#
Enhanced Advanced
If you prefer to make CRAN packages available for only specific dates, you can use a cran-snapshot
source. For example:
# List available snapshot dates (increase the count to see more options)
$ rspm list cran snapshots --count=25
# Create source from desired date:
$ rspm create source --name=cran-by-date --type=cran-snapshot --snapshot=2020-07-09
# Create a repository:
$ rspm create repo --name=cran --description='Access CRAN packages by date'
# Subscribe a repository to the cran-snapshot source:
$ rspm subscribe --repo=cran --source=cran-by-date
After completing these steps, the cran
repository is available in the web interface.
Use the update
command to change the snapshot:
# Move source to a newer snapshot
$ rspm update --source=cran-by-date --snapshot=2020-10-06 --commit
# Move source to an older snapshot
$ rspm update --source=cran-by-date --snapshot=2018-08-02 --commit
Serving Curated CRAN Subsets#
Enhanced Advanced
If you prefer to make only a subset of CRAN packages available, you can use a curated-cran
source. You can also specify which CRAN snapshot date to use. This snapshot date can be moved to older or newer versions of CRAN as needed.
# Create a curated-cran source from a specific starting date:
$ rspm create source --name=subset --type=curated-cran --snapshot=2024-04-16
Now that you've created the curated-cran source, include only the packages and versions you want to pull from CRAN. Curated CRAN sources support uploading requirements.txt
files to define what packages get included. As an example, let's look at a simple requirements.txt
file definition:
This fetches and installs:
A3
with versions greater than or equal to0.9.2
- Only
ggplot2
version3.4.4
along with suggested packages - All available versions of
plumber
Filtering by version is not supported for:
- Curated CRAN sources created prior to Package Manager version
2024.04
- Sources created using the
--strict
flag
In addition, the depends
, imports
, and linking-to
dependencies for each package is included.
rspm update
overwrite behavior
The requirements.txt
file declares the packages that are included in the source.
Packages are removed from the source after the update operation is performed if the packages are not in the source, are not included in the requirements.txt, and are not related packages.
With the requirements.txt
file defined above, we can now add the packages to the source:
# Do a dry-run to visualize the changes to the source before doing them
$ rspm update --source=subset --file-in=/path/to/requirements.txt --snapshot=2024-04-16
A preview of the changes is presented:
rspm update --source=subset --file-in=requirements.txt --snapshot=2024-04-16
Packages from 'requirements.txt' to update source 'subset' at CRAN snapshot date '2024-04-16':
Name Version Action
A3 1.0.0 add
askpass 1.2.0 add
backports 1.4.1 add
base64enc 0.1-3 add
<truncated>
If the output above looks correct, execute this command again with the --commit and --snapshot=2024-04-16 flags to update the source with the new set of packages.
Updating to the latest snapshot
To use the most recent snapshot available, omit the --snapshot
flag from the dryrun command.
The output provides information on all the packages that are being added and removed. The preview can be saved to a CSV file using the --csv-out
flag.
To commit the changes, repeat the command, adding the --commit
flag:
# Now commit the changes to the source:
$ rspm update --source=subset --file-in=/path/to/requirements.txt --snapshot=2024-04-16 --commit
The finalized contents of the source are then printed:
Successfully updated source 'subset' at CRAN snapshot date '2024-04-16' with the following packages from 'requirements.txt':
Name Version Action
A3 1.0.0 add
askpass 1.2.0 add
backports 1.4.1 add
base64enc 0.1-3 add
<truncated>
rspm update
overwrite behavior
Running rspm update
on a Curated CRAN source overwrites the source with only the packages defined in your requirements.txt
file. However, previous snapshots of the source are still available with a pinned repo URL.
Finally, be sure to subscribe a repository to the source to make the packages available to users:
# Create a repository:
$ rspm create repo --name=cran --type=r --description='Access Curated CRAN packages'
# Subscribe a repository to the curated-cran source:
$ rspm subscribe --repo=cran --source=subset
To update the source to a different snapshot date, use the update
command again:
# Update packages in a curated-cran source:
$ rspm update --source=subset --file-in=/path/to/requirements.txt --snapshot=2024-04-18 --commit
Curated CRAN snapshots
Curated CRAN snapshots allow you to set the Curated CRAN source to any date where a CRAN snapshot was taken on our servers. If you want to pin to a version of a package that no longer exists on CRAN, pin it to a date when the snapshot existed.
The snapshot date for non-strict sources can be moved either forward or backward in time.
For more information on Curated CRAN in Package Manager, see the Curated CRAN Sources section of the documentation.
Serving Curated PyPI Subsets#
Enhanced Advanced
If you prefer to make only a subset of PyPI packages available, you can use a curated-pypi
source. You can also specify which PyPI snapshot date to use:
# Create the curated-pypi source:
$ rspm create source --name=pypi-subset --type=curated-pypi
Now that the source has been created, it is time to include only the packages and versions you want from PyPI. Curated PyPI sources support uploading requirements.txt
files to define what packages get included. As an example, let's look at a simple requirements.txt
file definition:
This fetches and installs:
- All available versions of
shiny
- All versions of
tensorflow
greater than or equal to2.4.0
- Only
numpy
version1.24.2
With the requirements.txt
file defined above, we can now add the packages to the source:
# Do a dry-run to visualize the changes to the source before doing them
$ rspm update --source=pypi-subset --file-in=/path/to/requirements.txt --snapshot=2023-03-24
A preview of the changes is presented:
Packages from 'requirements.txt' to update source 'pypi-subset' at PyPI snapshot date '2023-03-24':
Name Version
numpy 1.24.2
shiny 0.1, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.2.6, 0.2.7, 0.2.8, 0.2.9, 0.2.10
tensorflow 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0, 2.6.0rc0, 2.6.0rc1, 2.6.1, 2.6.0rc2, 2.6.2, 2.6.3, 2.6.4, 2.6.5, 2.7.0, 2.7.0rc0, 2.7.0rc1, 2.7.1, 2.7.2, 2.7.3, 2.7.4, 2.8.0, 2.8.0rc0, 2.8.0rc1, 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.9.0, 2.9.0rc0, 2.9.0rc1, 2.9.1, 2.9.0rc2, 2.9.2, 2.9.3, 2.10.0, 2.10.0rc0, 2.10.0rc1, 2.10.1, 2.10.0rc2, 2.10.0rc3, 2.11.0, 2.11.0rc0, 2.11.0rc1, 2.11.1, 2.11.0rc2, 2.12.0, 2.12.0rc0, 2.12.0rc1
If the output above looks correct, execute this command again with the --commit flag to update the source with the new set of packages.
To commit the changes, repeat the command, adding the --commit
flag:
# Now commit the changes to the source:
$ rspm update --source=pypi-subset --file-in=/path/to/requirements.txt --snapshot=2023-03-24 --commit
The finalized contents of the source are then printed:
Successfully updated source 'pypi-subset' at PyPI snapshot date '2023-03-24' with the following packages from 'requirements.txt':
Name Version
numpy 1.24.2
shiny 0.1, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.2.6, 0.2.7, 0.2.8, 0.2.9, 0.2.10
tensorflow 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0, 2.6.0rc0, 2.6.0rc1, 2.6.1, 2.6.0rc2, 2.6.2, 2.6.3, 2.6.4, 2.6.5, 2.7.0, 2.7.0rc0, 2.7.0rc1, 2.7.1, 2.7.2, 2.7.3, 2.7.4, 2.8.0, 2.8.0rc0, 2.8.0rc1, 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.9.0, 2.9.0rc0, 2.9.0rc1, 2.9.1, 2.9.0rc2, 2.9.2, 2.9.3, 2.10.0, 2.10.0rc0, 2.10.0rc1, 2.10.1, 2.10.0rc2, 2.10.0rc3, 2.11.0, 2.11.0rc0, 2.11.0rc1, 2.11.1, 2.11.0rc2, 2.12.0, 2.12.0rc0, 2.12.0rc1
Note
Doing rspm update
on a Curated PyPI source overwrites the source with only the packages defined in your requirements.txt
file. However, previous snapshots of the source are still available with a pinned repo URL.
Finally, be sure to subscribe a repository to the source to make the packages available to users:
# Create a repository:
$ rspm create repo --name=pypi --type=python --description='Access Curated PyPI packages'
# Subscribe a repository to the curated-pypi source:
$ rspm subscribe --repo=pypi --source=pypi-subset
To update the source to a different snapshot date, use the update
command again:
# Update packages in a curated-pypi source:
$ rspm update --source=pypi-subset --file-in=/path/to/requirements.txt --snapshot=2023-03-17 --commit
Tip
Curated PyPI snapshots allow you to set the Curated PyPI source to any date where a PyPI snapshot has been taken on our servers. If you are trying to pin to a version of a package that doesn't exist on PyPI anymore, try pinning to a date when it existed.
For more information on Curated PyPI in Package Manager, see the Curated PyPI Sources section of the documentation.
Distributing Local R Packages#
Many teams have their own internally developed packages they want to make available. If your internal packages are tracked in Git, refer to the Building and serving R packages stored in Git section to build and publish them automatically.
If your internal packages are already built, they can be added to a Local source:
- Create the bundled source version of each package:
- Copy the resulting .tar.gz file(s) to the Package Manager server and add them to a local source:
# Create a local source:
$ rspm create source --name=internal-src --type=local
# Add each local package tar file to the source:
# The tar file must be rwx by the user running the CLI and the account running
# Package Manager (rstudio-pm by default)
$ rspm add --source=internal-src --path='/path/to/package_1.0.tar.gz'
# Create a repository:
$ rspm create repo --name=internal --description='Stable releases of our internal packages'
# Subscribe the repository to the source:
$ rspm subscribe --repo=internal --source=internal-src
Bulk Installing Packages#
Package Manager also supports installing multiple packages simultaneously in a single snapshot. This can be accomplished using one of the following options:
- Appending the paths together in a comma-seperated string with the
--path
flag - Passing in a CSV file with the
file-in
flag
The following is an example of the first method (appending all the paths together):
$ rspm add --source=internal-src --path='/path/to/package_1.0.tar.gz, /path/to/package2_1.1.tar.gz'
The second method supports passing in a list of packages in a CSV file. The format for the CSV looks like:
And the command to use the CSV file is:
Important
When using these methods, all packages will share the same --replace
value if the flag is provided. Make sure it is okay for all of the packages to be overwritten before running these commands.
Multiple Package Versions#
Package Manager automatically supports multiple versions of each package. When you are ready to add a new release of a package, run the add
command again:
Package Manager ensures that version 2.0 becomes the default for new installations. Previous package versions are retained in the repository's archive to be accessed through earlier snapshots.
Many internal packages also depend on public packages. To ensure that all the necessary dependencies are available, the recommended option is to create a repository that subscribes to both a local source and a source containing the public packages like CRAN or a curated source.
Building and serving R packages stored in Git#
Advanced
The configuration discussed above uses a local source and requires manual steps to add and update packages. If your organization uses Git to store internal R packages, then you can automate this process using a Git source.
Git sources require a valid R installation. For more information, see R Prerequisites.
# Create a Git source:
$ rspm create source --type=git --name=internal-src
# Create a Git builder, configured to surface tagged commits:
$ rspm create git-builder --url=https://bitbucket.example.com/r-pkg.git --source=internal-src --build-trigger=tags
# Check the status and learn more about the build:
$ rspm list git-builds --source=internal-src --name=r-pkg
# Read the build logs using the ID from the previous
# command output or from the "Activity" log in the UI:
$ rspm logs --transaction-id=[ID]
# Create a repository and subscribe it to the source:
$ rspm create repo --name=git --description='Stable releases of our internal packages'
$ rspm subscribe --source=internal-src --repo=git
Packages can be built using Git endpoints accessed via HTTP(s) or SSH URLs: https://github.com/user/repo.git
vs. git@github.com:user/repo.git
.
Note
For Git systems with non-standard ports, Package Manager also supports an SSH schema that can be used to specify the port. In practice this looks like: ssh://git@github.com:22/rstudio/rstudio.git
. See the SSH Protocol section of Git on the Server - The Protocols for more information.
If the Git URL uses SSH, then it requires an SSH key for authentication. In this case, import the key before using the create git-builder
command.
SSH keys are not required to use a passphrase, but a secure key with passphrase is recommended.
# Import the SSH key:
# passphrase file should just be text file with passphrase for key (avoid leaving in bash history):
$ rspm import ssh-key --name=read-r-pkg --path=/path/to/ssh/key --passphrase-path=/path/to/passphrase/file
# Optionally, remove the key from disk:
$ rm /path/to/ssh/key
# Create the Git builder
$ rspm create git-builder --url=user@bitbucket.example.com/r-pkg.git --source=internal-src --build-trigger=tags --credential=read-r-pkg
For more details on Git credential security or using HTTPS credentials, see Access Restricted Git Endpoints Using Git Credentials.
For more information on serving Git packages, see Building Git Packages.
Distributing Local Python Packages#
Along with PyPI mirroring support, Package Manager also supports adding internally-built Python packages. This guide will walk you through getting started with local Python packages in Package Manager.
Creating a new local Python source#
Similiar to the local
source for R packages, Package Manager has a local-python
source type that can be used for uploading your local Python packages. To create this source, run the following command:
# Create a local Python source:
$ rspm create source --name=local-python-src --type=local-python
<< Source 'local-python-src':
<< Type: Local Python
You can then create a python
repo and subscribe the local-python
source to it:
# Create a Python repository:
$ rspm create repo --name=local-python-repo --type=python --description='Access local Python packages'
<< Repository: local-python-repo - Access local Python packages - Python
# Subscribe the repository to the local Python source:
$ rspm subscribe --repo=local-python-repo --source=local-python-src
<< Repository: local-python-repo
<< Sources:
<< --local-python-src (Local Python)
Uploading Python Packages#
Once the source and repository have been created, you can start uploading Python packages to Package Manager. You can use the rspm add
command and pass either a comma-separated list of files or simply pass the path to a directory containing the Python packages you want to upload:
# Upload a single package:
$ rspm add --source=local-python-src --path=/path/to/package-1.0.0.tar.gz
# Upload a signed package:
$ rspm add --source=local-python-src --path=/path/to/example-0.0.3-py3-none-any.whl,/path/to/example-0.0.3-py3-none-any.whl.asc
# Upload multiple packages from a directory:
$ rspm add --source=local-python-src --path=/path/to/directory
# Upload multiple packages from a directory, keeping any existing packages:
$ rspm add --source=local-python-src --path=/path/to/directory --succeed-on-existing
# Upload multiple packages from a directory, replacing any existing packages:
$ rspm add --source=local-python-src --path=/path/to/directory --replace
Note
When uploading multiple packages from a directory, any signature files (*.asc
) will be automatically included with their respective package files.
Remote Publishing#
Advanced
Local Python sources also support remote token authentication. You can use remote authentication to upload Python packages with either the CLI or with twine
.
For more information, see the Remote Publishing documentation.
Building and serving Python packages stored in Git#
Advanced
If your organization uses Git to store internal Python packages, then you can automate the process of building and uploading using a Git Python source.
Git Python sources require a valid Python installation. For more information, see Python Prerequisites.
Once Python is installed, two modules are also required for building the package: build
and virtualenv
. To install these, run:
# Install the necessary dependencies
$ pip install build virtualenv
# Verify they were properly installed
$ python -m build --version
$ python -m virtualenv --version
If Python and the required modules are not installed, Git builders for Python packages will be disabled. When they are detected, a Python Git builder can be created:
# Create a Git Python source:
$ rspm create source --type=git-python --name=internal-python-src
# Create a Python Git builder, configured to surface tagged commits:
$ rspm create git-builder --url=https://bitbucket.example.com/python-pkg.git --source=internal-python-src --build-trigger=tags --name=python-pkg
# Check the status and learn more about the build:
$ rspm list git-builds --source=internal-python-src --name=python-pkg
# Read the build logs using the ID from the previous
# command output or from the "Activity" log in the UI:
$ rspm logs --transaction-id=[ID]
# Create a repository and subscribe it to the source:
$ rspm create repo --name=git-python --description='Stable releases of our internal Python packages' --type=python
$ rspm subscribe --source=internal-python-src --repo=git-python
Packages can be built using Git endpoints accessed via HTTP(s) or SSH URLs: https://github.com/user/repo.git
vs. git@github.com:user/repo.git
.
Note
For Git systems with non-standard ports, Package Manager also supports an SSH schema that can be used to specify the port. In practice this looks like: ssh://git@github.com:22/rstudio/rstudio.git
. See the SSH Protocol section of Git on the Server - The Protocols for more information.
If the Git URL uses SSH, then it requires an SSH key for authentication. In this case, import the key before using the create git-builder
command.
SSH keys are not required to use a passphrase, but a secure key with passphrase is recommended.
# Import the SSH key:
# passphrase file should just be text file with passphrase for key (avoid leaving in bash history):
$ rspm import ssh-key --name=read-python-pkg --path=/path/to/ssh/key --passphrase-path=/path/to/passphrase/file
# Optionally, remove the key from disk:
$ rm /path/to/ssh/key
# Create the Python Git builder
$ rspm create git-builder --url=user@bitbucket.example.com/python-pkg.git --source=internal-python-src --build-trigger=tags --name=python-pkg --credential=read-python-pkg
For more details on Git credential security or using HTTPS credentials, see Access Restricted Git Endpoints Using Git Credentials.
For more information on serving Git packages, see Building Git Packages.
Serving User Binary Packages#
While Package Manager provides pre-built binaries for CRAN source packages, users may want to upload their own binaries to speed up install times, reduce the need for build-time dependencies, and improve reproducibility across environments.
For more information on binary packages, see Serving Package Binaries.
Tip
R binaries target specific R versions, operating systems, CPU architectures, and system dependency versions. Package Manager will handle multiple binaries for a single package and will automatically detect the R version and CPU architecture from the DESCRIPTION
file.
Local Binary Packages#
You can add your own pre-compiled binary packages to local sources. After adding a source package, you can supplement the source with additional binary packages:
- Create the bundled source version of each package
- Create pre-compiled binary packages for the R version and distributions you need. See Adding Local and Git Binaries for more information about R binary packages.
# Build the source package, package_1.0.tar.gz
$ R CMD build /path/to/package
# Build the binary package, package_1.0-binary.tar.gz (Linux) or package_1.0.zip (Windows)
$ R CMD INSTALL --build package_1.0.tar.gz
- Install the source and binary packages:
# Create a local source:
$ rspm create source --name=internal-src
# Add the local package tar file to the source:
$ rspm add --source=internal-src --path='/path/to/package_1.0.tar.gz'
# List available binary distributions
$ rspm list distributions
<< Binary distributions:
<< jammy
<< ...
<< windows
# Add the precompiled binary packages to the source:
$ rspm add binary --source=internal-src --distribution=jammy --path='/path/to/package-binary.tar.gz'
$ rspm add binary --source=internal-src --distribution=windows --path='/path/to/package-binary.zip'
- Subscribe the source to a new (or existing) repository:
# Create a repository:
$ rspm create repo --name=internal --description='Stable releases of our internal packages with binaries'
# Subscribe the repository to the source:
$ rspm subscribe --repo=internal --source=internal-src
Git Binary Packages#
Advanced
You can add your own precompiled binary packages to Git sources. After building a Git package, you can supplement the source with additional binary packages. See Building and serving R packages stored in Git for an example.
Note
Git sources for R packages automatically assign unique versions to each source package when building packages based on commits to a branch. The automatic version includes a suffix that is based on the current timestamp, which makes it difficult to upload precompiled binary packages for the Git source. When you wish to upload binary packages for a Git source, we recommend configuring the Git builder to use tags. See Commits vs Tags for more details.
- Create the Git package
# Create a Git source:
$ rspm create source --type=git --name=internal-src
# Create a Git builder, configured to surface tagged commits:
$ rspm create git-builder --url=https://bitbucket.example.com/r-pkg.git --source=internal-src --build-trigger=tags
# Create a repository and subscribe it to the source:
$ rspm create repo --name=git --description='Stable releases of our internal packages'
$ rspm subscribe --source=internal-src --repo=git
- Create precompiled binary packages for the R version and distributions you need. See Adding Local and Git Binaries for more information about R binary packages.
# Build the source package, package_1.0.tar.gz
$ R CMD build /path/to/package
# Build the binary package, package_1.0-binary.tar.gz (Linux) or package_1.0.zip (Windows)
$ R CMD INSTALL --build package_1.0.tar.gz
- Install the binary packages:
# List available binary distributions
$ rspm list distributions
<< Binary distributions:
<< jammy
<< ...
<< windows
# Add the precompiled binary packages to the Git source:
$ rspm add binary --source=internal-src --distribution=jammy --path='/path/to/package-binary.tar.gz'
$ rspm add binary --source=internal-src --distribution=windows --path='/path/to/package-binary.zip'
Binary Packages using the CLI Remotely#
Advanced
You can use the Package Manager CLI remotely to add source and binary packages.
On the Package Manager server, use the CLI to generate an API token with access to the correct sources:
$ rspm create token --description="user A" --sources=local-src --expires=30d --scope=sources:write
<< Generated an access token. Be sure to record this token immediately since you will not be able to retrieve it later.
<< eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJwYWNrYWdlbWFuYWdlciIsImV4cCI6MTY1ODU4MjA3OCwianRpIjoiYmM5ZTg1NGYtNGNlNy00Zjc4LTlhMmMtZDliYzRlYTQ0NGVkIiwiaWF0IjoxNjU1OTkwMDc4LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjQyNDIvIiwic2NvcGVzIjp7InNvdXJjZXMiOiIzYTI4NjFhYi0xNWYwLTRjM2MtODZlMy0xNjNkMTY0ZDE0ZDYifX0.BWJXLworo44Nvfrh5a2hm_NIqgUoXTLjQlxyy7uaSWk
On the remote machine, export the correct environment variables and download the CLI:
$ export PACKAGEMANAGER_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJwYWNrYWdlbWFuYWdlciIsImV4cCI6MTY1ODU4MjA3OCwianRpIjoiYmM5ZTg1NGYtNGNlNy00Zjc4LTlhMmMtZDliYzRlYTQ0NGVkIiwiaWF0IjoxNjU1OTkwMDc4LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjQyNDIvIiwic2NvcGVzIjp7InNvdXJjZXMiOiIzYTI4NjFhYi0xNWYwLTRjM2MtODZlMy0xNjNkMTY0ZDE0ZDYifX0.BWJXLworo44Nvfrh5a2hm_NIqgUoXTLjQlxyy7uaSWk
$ export PACKAGEMANAGER_ADDRESS=<https://my-rspm-server.org:4443>
# Download the CLI
$ curl -fLOJH "Authorization: Bearer ${PACKAGEMANAGER_TOKEN}" "${PACKAGEMANAGER_ADDRESS}/__api__/download"
$ chmod +x rspm
$ export PACKAGEMANAGER_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJwYWNrYWdlbWFuYWdlciIsImV4cCI6MTY1ODU4MjA3OCwianRpIjoiYmM5ZTg1NGYtNGNlNy00Zjc4LTlhMmMtZDliYzRlYTQ0NGVkIiwiaWF0IjoxNjU1OTkwMDc4LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjQyNDIvIiwic2NvcGVzIjp7InNvdXJjZXMiOiIzYTI4NjFhYi0xNWYwLTRjM2MtODZlMy0xNjNkMTY0ZDE0ZDYifX0.BWJXLworo44Nvfrh5a2hm_NIqgUoXTLjQlxyy7uaSWk
$ export PACKAGEMANAGER_ADDRESS=<https://my-rspm-server.org:4443>
# Download the CLI
$ curl -fLOJH "Authorization: Bearer ${PACKAGEMANAGER_TOKEN}" "${PACKAGEMANAGER_ADDRESS}/__api__/download?os=darwin"
$ chmod +x rspm
$Env:PACKAGEMANAGER_TOKEN = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJwYWNrYWdlbWFuYWdlciIsImV4cCI6MTY1ODU4MjA3OCwianRpIjoiYmM5ZTg1NGYtNGNlNy00Zjc4LTlhMmMtZDliYzRlYTQ0NGVkIiwiaWF0IjoxNjU1OTkwMDc4LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjQyNDIvIiwic2NvcGVzIjp7InNvdXJjZXMiOiIzYTI4NjFhYi0xNWYwLTRjM2MtODZlMy0xNjNkMTY0ZDE0ZDYifX0.BWJXLworo44Nvfrh5a2hm_NIqgUoXTLjQlxyy7uaSWk"
$Env:PACKAGEMANAGER_ADDRESS = "https://my-rspm-server.org:4443"
# Download the CLI
$urlCLI = "$Env:PACKAGEMANAGER_ADDRESS/__api__/download?os=windows"
$outputCLI = "rspm.exe"
$wc = New-Object System.Net.WebClient
$wc.Headers['Authorization'] = "Bearer $Env:PACKAGEMANAGER_TOKEN"
$wc.DownloadFile($urlCLI, $outputCLI)
# Alternatively, on Windows 10 (v1803+), you can download the CLI with curl
curl -OutFile rspm.exe -Headers @{ Authorization = "Bearer $Env:PACKAGEMANAGER_TOKEN"} "$Env:PACKAGEMANAGER_ADDRESS/__api__/download?os=windows"
After the environment variables are set, the remote machine can use CLI commands that support remote use: