github.com/zendesk/apt-s3
go get github.com/zendesk/apt-s3

github.com/zendesk/apt-s3

apt method for private S3 buckets

by zendesk

v0.0.0-20200810150839-07aa44769983 (see all)License:Apache-2.0
go get github.com/zendesk/apt-s3
Readme

apt-s3

repo-checks

apt-s3 is an APT Method Interface written in Go to use a private S3 bucket as an apt repository on Debian based systems. Similar projects exist, but they all have their caveats:

  • Many are completely unmaintained
  • Most require python and some require additional pip packages
  • Some only use the default AWS authentication methods
    • This means any application specific credentials in a Docker container must also have access to the S3 bucket or apt breaks entirely
  • Most set the region globally so they only support a single S3 region at a time
  • Some place the API keys in the S3 URI
    • This means they are leaked every time apt-get update or apt-get install is run
  • Some do not use the AWS SDK
  • None of them expose an interactive component for downloading files

Installation

The only requirement for apt-s3 is the ca-certificates package and its dependencies.

Installation is as easy as downloading the binary or deb package from our releases page.

Package Installation

Download the package and install it with dpkg -i /path/to/package.deb. If you see the error message below simply run apt-get install -f to fix it.

dpkg: dependency problems prevent configuration of apt-s3:
 apt-s3 depends on ca-certificates; however:
  Package ca-certificates is not installed.

Binary Installation

Download the binary and move it to /usr/lib/apt/methods/s3.

Usage

Simply create an apt list file in the proper format to start using apt-s3 with apt.

export BUCKET_NAME=my-s3-bucket
export BUCKET_REGION=us-east-1

echo "deb s3://${BUCKET_NAME}.s3-${BUCKET_REGION}.amazonaws.com/ stable main" > /etc/apt/source.list.d/s3bucket.list"

Credentials File

/etc/apt/s3creds is checked before using the default AWS credential methods. The file has a format similar to ~/.aws/credentials, but profiles are ignored.

aws_access_key_id     = foo
aws_secret_access_key = foobar123
aws_session_token     = not-normally-needed

Interactive Usage

To download a file using apt-s3 simply use the -download flag. Run apt-s3 -help for usage info.

export BUCKET_NAME=my-s3-bucket
export BUCKET_REGION=us-east-1

apt-s3 -download s3:/${BUCKET_NAME}.s3-${BUCKET_REGION}.amazonaws.com/file -path /tmp/file

Building

Use the Makefile to build the binary and .deb package (requires nfpm to be installed and in the $PATH).

$ make

Releasing a New Version

To release a new version you will need a few things:

  1. Write access to this repo
  2. A personal access token
  3. nfpm installed and in the $PATH

Once everything is set up follow these steps to create a release and upload assets:

export GITHUB_TOKEN=<your personal access token>
# bumping the version programatically can be easily done with `awk`
export VERSION=$(git describe --tags | awk -F. -v OFS=. '{ $3++ } 1') # use $1 for major/$2 for minor/$3 for patch
make VERSION=$VERSION release

Contributing

Improvements are always welcome. Please follow these steps to contribute:

  1. Fork repo
  2. Submit a Pull Request with a detailed explanation of changes
  3. Receive a 👍 from a core team member
  4. Core team will merge your changes

License

Use of this software is subject to important terms and conditions as set forth in the LICENSE file.

GitHub Stars

12

LAST COMMIT

2yrs ago

MAINTAINERS

0

CONTRIBUTORS

5

OPEN ISSUES

0

OPEN PRs

0
VersionTagPublished
v0.0.0-20200810150839-07aa44769983
1yr ago
v0.0.0-20190415130422-fe8b25820a99
2yrs ago
No alternatives found
No tutorials found
Add a tutorial