Compare commits

..

63 Commits
0.2.0 ... 0.4.4

Author SHA1 Message Date
2f3ba1335c Update docs for #15
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-05 12:02:50 +00:00
24e065965f Update path for secrets for e2e tests
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-05 12:00:43 +00:00
fd2ee55f9f Update e2e tests for secrets folder
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-05 12:00:43 +00:00
d135999d3b Set working directory for faasd / faas-containerd
* faasd writes secrets to wd + /secrets/*
* faas-containerd is passed a custom path to use to load the
secrets

Both services gain their work /run/ folders for temporary and
working files. Tested on RPi3 e2e with faasd install.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-05 12:00:43 +00:00
3068d03279 Set wd to /run/faasd
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-05 12:00:43 +00:00
fd4f53fe15 Update faas-containerd.service 2020-01-05 09:23:19 +00:00
4b93ccba3f Debug statement for CI
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-05 08:56:29 +00:00
e6b814fd60 Add basic_auth env variable to faas-containerd.service
Fixes #11
Having this env variable set enables basic authentication for faas-containerd.

Signed-off-by: Mark Sharpley <mcs94@cam.ac.uk>
2020-01-05 08:32:01 +00:00
06890cddb9 Tweak e2e tests
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 19:41:04 +00:00
40da0a35c3 Tweak e2e tests
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 19:35:01 +00:00
0c0d05b2ea Tweak e2e tests
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 19:12:10 +00:00
af46f34003 Tweak e2e tests
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 19:05:17 +00:00
2f7269fc97 Tweak e2e tests
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:58:37 +00:00
1f56c39675 Tweak e2e tests
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:53:03 +00:00
7152b170bb Tweak e2e tests
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:47:25 +00:00
47955954eb Tweak e2e tests
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:43:46 +00:00
7f672f006a Tweak e2e tests
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:29:46 +00:00
20ec76bf74 Run faasd from usr local bin
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:25:27 +00:00
04d1688bfb Repurpose dist binary for e2e test
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:21:48 +00:00
a8f514f7d6 Show pwd in prepare-test step
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:13:12 +00:00
502d3fdecc Show working dir
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:10:44 +00:00
5d098c9cb7 Invoke async in e2e tests
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:06:31 +00:00
0935dc6867 Use GOPATH from env
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:04:24 +00:00
e77d05ec94 Test e2e
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 18:02:28 +00:00
7ab69b5317 Add e2e test prep
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 17:59:11 +00:00
098baba7cc Add unit test for proxy and shutdown channel
* Proxy has initial unit test and more can be added
* Shutdown channel and cancellation added for proper shutdown of
the proxy

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 12:06:53 +00:00
9d688b9ea6 Bump faasd version
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-02 18:40:12 +00:00
1458a2f2fb Add Caddyfile to backlog
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-01 12:35:33 +00:00
c18a038062 Generate basic-auth password in install command
* Required so that faas-containerd can start independently of
faasd.
* Extracts common mount path const for mounted secrets

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-01 12:33:11 +00:00
af0555a85b Add tutorial and give other options for Linux users
Closes: #9
Closes: #8

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-01 12:17:58 +00:00
2ff8646669 Add .github
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-01 12:10:02 +00:00
d785bebf4c Copy headers in both directions in proxy
* Issue was detected whilst testing 0.4.0 from @Waterdrips which
added basic auth, but the header was not being propagated.
* This code is tested in OpenFaaS already, but unit tests will
be added retrospectively.
* Proxy now reads the gateway URL via a channel instead of from
a file to make unit testing easier.

Basic auth now works as expected with faas-cli login / list.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-31 18:20:43 +00:00
17845457e2 Alteration for version passing
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-31 12:30:14 +00:00
300d8b082a Pass version from main
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-31 12:30:14 +00:00
17a5e2c625 Extract file for version command
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-31 12:18:27 +00:00
f0172e618a Move basic auth note to common section
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-30 20:48:27 +00:00
61e2d16c3e Add Basic Auth to the gateway
Add and enable basic auth to the gateway. This allows users to
put their gateway on the internet and expose it to public networks
without anyone being able to control their deployments.

Added information to the README that allows users to get their
gatewau basic auth password and username

Signed-off-by: Alistair Hey <alistair@heyal.co.uk>
2019-12-30 20:45:00 +00:00
ae0753a6d9 Update faasd version
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-29 13:16:16 +00:00
19a769b7da Update proxy print message
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-29 13:02:03 +00:00
48237e0b3c Don't follow redirects
Required for functioning proxy

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-29 12:59:39 +00:00
306313ed9a Proxy from faasd to gateway
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-29 12:46:32 +00:00
ff0cccf0dc Add proxy to faasd up
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-29 12:46:32 +00:00
52baca9d17 Update download location of Go
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-29 12:00:54 +00:00
f76432f60a Add use of template to Install command
The template name wasnt used, so the command gave an error saying
that no template was used.

Signed-off-by: Alistair Hey <alistair@heyal.co.uk>
2019-12-29 11:28:16 +00:00
38f26b213f Clear snapshot when container doesn't exist
This clears up a scenario where a container can be deleted but
its snapshot is not.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 20:01:01 +00:00
6c3fe813fd Extract PrepareImage
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 19:09:42 +00:00
13d28bd2db Extract Service struct
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 18:32:13 +00:00
f3f6225674 Bump faasd version
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 16:46:53 +00:00
e4ed9e5b91 Fix typo
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 16:46:08 +00:00
5a28f3e231 Add error handling for when template not found
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 16:41:06 +00:00
a042be5477 Mention where to run the install
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 16:38:12 +00:00
6230d3504e Run faasd from /usr/local/bin
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 16:09:02 +00:00
4ba3ec3b64 Make pre-reqs more obvious
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-27 09:28:04 +00:00
ea9386d285 Update backlog
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-27 09:26:30 +00:00
03500c5649 Update backlog
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-27 09:25:21 +00:00
867f8459b0 Define pre-reqs
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-27 09:21:56 +00:00
6737712b28 Update version in README
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-27 09:16:40 +00:00
832893998d Bump gateway
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-25 22:11:56 +00:00
1732566748 Explicitly turn off write_debug for queue-worker
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-24 19:50:43 +00:00
3b512f979c Fix arm switch
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-24 14:34:32 +00:00
456e56342e Use k3sup to determine client arch
* Set image suffix for OpenFaaS

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-24 14:21:47 +00:00
d19d8998d8 Fix vendoring for go-execute
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-24 10:30:24 +00:00
376c8e5d7b Update to latest release
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-24 10:11:54 +00:00
30 changed files with 1424 additions and 234 deletions

1
.github/CODEOWNERS vendored Normal file
View File

@ -0,0 +1 @@
@alexellis

41
.github/ISSUE_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,41 @@
<!--- Provide a general summary of the issue in the Title above -->
## Expected Behaviour
<!--- If you're describing a bug, tell us what should happen -->
<!--- If you're suggesting a change/improvement, tell us how it should work -->
## Current Behaviour
<!--- If describing a bug, tell us what happens instead of the expected behavior -->
<!--- If suggesting a change/improvement, explain the difference from current behavior -->
## Possible Solution
<!--- Not obligatory, but suggest a fix/reason for the bug, -->
<!--- or ideas how to implement the addition or change -->
## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant -->
1.
2.
3.
4.
## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
## Your Environment
* OS and architecture:
* Versions:
```sh
go version
containerd -version
uname -a
cat /etc/os-release
```

40
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,40 @@
<!--- Provide a general summary of your changes in the Title above -->
## Description
<!--- Describe your changes in detail -->
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here. -->
- [ ] I have raised an issue to propose this change **this is required**
## How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran to -->
<!--- see how your change affects other areas of the code, etc. -->
## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
Commits:
- [ ] I've read the [CONTRIBUTION](https://github.com/openfaas/faas/blob/master/CONTRIBUTING.md) guide
- [ ] My commit message has a body and describe how this was tested and why it is required.
- [ ] I have signed-off my commits with `git commit -s` for the Developer Certificate of Origin (DCO)
Code:
- [ ] My code follows the code style of this project.
- [ ] I have added tests to cover my changes.
Docs:
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.

4
.gitignore vendored
View File

@ -1,3 +1,7 @@
/faasd
hosts
/resolv.conf
.idea/
basic-auth-user
basic-auth-password

View File

@ -1,9 +1,19 @@
sudo: required
language: go
go:
- '1.12'
addons:
apt:
packages:
- runc
script:
- make dist
- make prepare-test
- make test-e2e
deploy:
provider: releases
api_key:

28
Gopkg.lock generated
View File

@ -37,6 +37,22 @@
pruneopts = "UT"
revision = "9e921883ac929bbe515b39793ece99ce3a9d7706"
[[projects]]
digest = "1:74860eb071d52337d67e9ffd6893b29affebd026505aa917ec23131576a91a77"
name = "github.com/alexellis/go-execute"
packages = ["pkg/v1"]
pruneopts = "UT"
revision = "961405ea754427780f2151adff607fa740d377f7"
version = "0.3.0"
[[projects]]
digest = "1:6076d857867a70e87dd1994407deb142f27436f1293b13e75cc053192d14eb0c"
name = "github.com/alexellis/k3sup"
packages = ["pkg/env"]
pruneopts = "UT"
revision = "f9a4adddc732742a9ee7962609408fb0999f2d7b"
version = "0.7.1"
[[projects]]
digest = "1:386ca0ac781cc1b630b3ed21725759770174140164b3faf3810e6ed6366a970b"
name = "github.com/containerd/containerd"
@ -257,6 +273,14 @@
revision = "ba968bfe8b2f7e042a574c888954fccecfa385b4"
version = "v0.8.1"
[[projects]]
digest = "1:044c51736e2688a3e4f28f72537f8a7b3f9c188fab4477d5334d92dfe2c07ed5"
name = "github.com/sethvargo/go-password"
packages = ["password"]
pruneopts = "UT"
revision = "07c3d521e892540e71469bb0312866130714c038"
version = "v0.1.3"
[[projects]]
digest = "1:fd61cf4ae1953d55df708acb6b91492d538f49c305b364a014049914495db426"
name = "github.com/sirupsen/logrus"
@ -440,7 +464,8 @@
analyzer-name = "dep"
analyzer-version = 1
input-imports = [
"github.com/alexellis/go-execute",
"github.com/alexellis/go-execute/pkg/v1",
"github.com/alexellis/k3sup/pkg/env",
"github.com/containerd/containerd",
"github.com/containerd/containerd/cio",
"github.com/containerd/containerd/containers",
@ -449,6 +474,7 @@
"github.com/containerd/containerd/oci",
"github.com/morikuni/aec",
"github.com/opencontainers/runtime-spec/specs-go",
"github.com/sethvargo/go-password/password",
"github.com/spf13/cobra",
"github.com/vishvananda/netlink",
"github.com/vishvananda/netns",

View File

@ -10,6 +10,9 @@
name = "github.com/spf13/cobra"
version = "0.0.5"
[[constraint]]
name = "github.com/alexellis/k3sup"
version = "0.7.1"
[[constraint]]
name = "github.com/alexellis/go-execute"
@ -18,3 +21,7 @@
[prune]
go-tests = true
unused-packages = true
[[constraint]]
name = "github.com/sethvargo/go-password"
version = "0.1.3"

View File

@ -1,6 +1,8 @@
Version := $(shell git describe --tags --dirty)
GitCommit := $(shell git rev-parse HEAD)
LDFLAGS := "-s -w -X pkg.Version=$(Version) -X pkg.GitCommit=$(GitCommit)"
LDFLAGS := "-s -w -X main.Version=$(Version) -X main.GitCommit=$(GitCommit)"
CONTAINERD_VER := 1.3.2
FAASC_VER := 0.4.0
.PHONY: all
all: local
@ -13,3 +15,34 @@ dist:
CGO_ENABLED=0 GOOS=linux go build -ldflags $(LDFLAGS) -a -installsuffix cgo -o bin/faasd
CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 go build -ldflags $(LDFLAGS) -a -installsuffix cgo -o bin/faasd-armhf
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags $(LDFLAGS) -a -installsuffix cgo -o bin/faasd-arm64
.PHONY: prepare-test
prepare-test:
curl -sLSf https://github.com/containerd/containerd/releases/download/v$(CONTAINERD_VER)/containerd-$(CONTAINERD_VER).linux-amd64.tar.gz > /tmp/containerd.tar.gz && sudo tar -xvf /tmp/containerd.tar.gz -C /usr/local/bin/ --strip-components=1
curl -SLfs https://raw.githubusercontent.com/containerd/containerd/v1.3.2/containerd.service | sudo tee /etc/systemd/system/containerd.service
sudo systemctl daemon-reload && sudo systemctl start containerd
sudo curl -fSLs "https://github.com/genuinetools/netns/releases/download/v0.5.3/netns-linux-amd64" --output "/usr/local/bin/netns" && sudo chmod a+x "/usr/local/bin/netns"
sudo /sbin/sysctl -w net.ipv4.conf.all.forwarding=1
sudo curl -sSLf "https://github.com/alexellis/faas-containerd/releases/download/$(FAASC_VER)/faas-containerd" --output "/usr/local/bin/faas-containerd" && sudo chmod a+x "/usr/local/bin/faas-containerd" || :
sudo cp $(GOPATH)/src/github.com/alexellis/faasd/bin/faasd /usr/local/bin/
cd $(GOPATH)/src/github.com/alexellis/faasd/ && sudo /usr/local/bin/faasd install
sudo systemctl status -l containerd --no-pager
sudo journalctl -u faas-containerd --no-pager
sudo systemctl status -l faas-containerd --no-pager
sudo systemctl status -l faasd --no-pager
curl -sSLf https://cli.openfaas.com | sudo sh
sleep 120 && sudo journalctl -u faasd --no-pager
.PHONY: test-e2e
test-e2e:
sudo cat /run/faasd/secrets/basic-auth-password | /usr/local/bin/faas-cli login --password-stdin
/usr/local/bin/faas-cli store deploy figlet --env write_timeout=1s --env read_timeout=1s
sleep 2
/usr/local/bin/faas-cli list -v
uname | /usr/local/bin/faas-cli invoke figlet
uname | /usr/local/bin/faas-cli invoke figlet --async
sleep 10
/usr/local/bin/faas-cli list -v
/usr/local/bin/faas-cli remove figlet
sleep 3
/usr/local/bin/faas-cli list

104
README.md
View File

@ -24,66 +24,104 @@ You can use the standard [faas-cli](https://github.com/openfaas/faas-cli) with f
* `faas describe`
* `faas deploy --update=true --replace=false`
* `faas invoke`
* `faas invoke --async`
Other operations are pending development in the provider.
### Pre-reqs
* Linux - ideally Ubuntu, which is used for testing.
* Linux
PC / Cloud - any Linux that containerd works on should be fair game, but faasd is tested with Ubuntu 18.04
For Raspberry Pi Raspbian Stretch or newer also works fine
For MacOS users try [multipass.run](https://multipass.run) or [Vagrant](https://www.vagrantup.com/)
For Windows users, install [Git Bash](https://git-scm.com/downloads) along with multipass or vagrant. You can also use WSL1 or WSL2 which provides a Linux environment.
* Installation steps as per [faas-containerd](https://github.com/alexellis/faas-containerd) for building and for development
* [netns](https://github.com/genuinetools/netns/releases) binary in `$PATH`
* [containerd v1.3.2](https://github.com/containerd/containerd)
* [faas-cli](https://github.com/openfaas/faas-cli) (optional)
## Backlog
* Use CNI to create network namespaces and adapters
* Inject / manage IPs between core components for service to service communication - i.e. so Prometheus can scrape the OpenFaaS gateway
* Monitor and restart any of the core components, if they crash
* Configure `basic_auth` to protect the OpenFaaS gateway and faas-containerd HTTP API
* Self-install / create systemd service on start-up using [go-systemd](https://github.com/coreos/go-systemd)
* Bundle/package/automate installation of containerd - [see bootstrap from k3s](https://github.com/rancher/k3s)
* Create [faasd.service](https://github.com/rancher/k3s/blob/master/k3s.service)
Pending:
* [ ] Use CNI to create network namespaces and adapters
* [ ] Monitor and restart any of the core components at runtime if the container stops
* [ ] Bundle/package/automate installation of containerd - [see bootstrap from k3s](https://github.com/rancher/k3s)
* [ ] Provide ufw rules / example for blocking access to everything but a reverse proxy to the gateway container
* [ ] Provide [simple Caddyfile example](https://blog.alexellis.io/https-inlets-local-endpoints/) in the README showing how to expose the faasd proxy on port 80/443 with TLS
## Hacking
Done:
* [x] Inject / manage IPs between core components for service to service communication - i.e. so Prometheus can scrape the OpenFaaS gateway - done via `/etc/hosts` mount
* [x] Add queue-worker and NATS
* [x] Create faasd.service and faas-containerd.service
* [x] Self-install / create systemd service via `faasd install`
* [x] Restart containers upon restart of faasd
* [x] Clear / remove containers and tasks with SIGTERM / SIGINT
* [x] Determine armhf/arm64 containers to run for gateway
* [x] Configure `basic_auth` to protect the OpenFaaS gateway and faas-containerd HTTP API
* [x] Setup custom working directory for faasd `/run/faasd/`
## Tutorial: Get started on armhf / Raspberry Pi
You can run this tutorial on your Raspberry Pi, or adapt the steps for a regular Linux VM/VPS host.
* [faasd - lightweight Serverless for your Raspberry Pi](https://blog.alexellis.io/faasd-for-lightweight-serverless/)
## Hacking (build from source)
First run faas-containerd
```sh
cd $GOPATH/src/github.com/alexellis/faas-containerd
go build && sudo ./faas-containerd
# You'll need to install containerd and its pre-reqs first
# https://github.com/alexellis/faas-containerd/
sudo ./faas-containerd
```
Then run faasd, which brings up the gateway and Prometheus as containers
```sh
cd $GOPATH/src/github.com/alexellis/faasd
go build && sudo ./faasd
go build
# Install with systemd
# sudo ./faasd install
# Or run interactively
# sudo ./faasd up
```
Or get from binaries:
### Build and run faas-containerd
### Build and run (binaries)
```sh
# For x86_64
sudo curl -fSLs "https://github.com/alexellis/faasd/releases/download/0.1.3/faasd" \
sudo curl -fSLs "https://github.com/alexellis/faasd/releases/download/0.4.2/faasd" \
-o "/usr/local/bin/faasd" \
&& sudo chmod a+x "/usr/local/bin/faasd"
# armhf
sudo curl -fSLs "https://github.com/alexellis/faasd/releases/download/0.1.3/faasd-armhf" \
sudo curl -fSLs "https://github.com/alexellis/faasd/releases/download/0.4.2/faasd-armhf" \
-o "/usr/local/bin/faasd" \
&& sudo chmod a+x "/usr/local/bin/faasd"
# arm64
sudo curl -fSLs "https://github.com/alexellis/faasd/releases/download/0.1.3/faasd-arm64" \
sudo curl -fSLs "https://github.com/alexellis/faasd/releases/download/0.4.2/faasd-arm64" \
-o "/usr/local/bin/faasd" \
&& sudo chmod a+x "/usr/local/bin/faasd"
```
### At run-time
Look in `hosts` in the current working folder to get the IP for the gateway or Prometheus
Look in `hosts` in the current working folder or in `/run/faasd/` to get the IP for the gateway or Prometheus
```sh
127.0.0.1 localhost
@ -91,20 +129,39 @@ Look in `hosts` in the current working folder to get the IP for the gateway or P
172.19.0.2 prometheus
172.19.0.3 gateway
172.19.0.4 nats
172.19.0.5 queue-worker
```
Since faas-containerd uses containerd heavily it is not running as a container, but as a stand-alone process. Its port is available via the bridge interface, i.e. netns0.
Now go to the gateway's IP address as shown above on port 8080, i.e. http://172.19.0.3:8080 - you can also use this address to deploy OpenFaaS Functions via the `faas-cli`.
* Prometheus will run on the Prometheus IP plus port 8080 i.e. http://172.19.0.2:9090/targets
* faas-containerd runs on 172.19.0.1:8081
* Now go to the gateway's IP address as shown above on port 8080, i.e. http://172.19.0.3:8080 - you can also use this address to deploy OpenFaaS Functions via the `faas-cli`.
* basic-auth
You will then need to get the basic-auth password, it is written to `/run/faasd/secrets/basic-auth-password` if you followed the above instructions.
The default Basic Auth username is `admin`, which is written to `/run/faasd/secrets/basic-auth-user`, if you wish to use a non-standard user then create this file and add your username (no newlines or other characters)
#### Installation with systemd
* `faasd install` - install faasd and containerd with systemd, this must be run from `$GOPATH/src/github.com/alexellis/faasd`
* `journalctl -u faasd` - faasd systemd logs
* `journalctl -u faas-containerd` - faas-containerd systemd logs
### Appendix
Removing containers:
```sh
echo faas-containerd gateway prometheus |xargs sudo ctr task rm -f
echo faas-containerd gateway prometheus | xargs sudo ctr task rm -f
echo faas-containerd gateway prometheus |xargs sudo ctr container rm
echo faas-containerd gateway prometheus | xargs sudo ctr container rm
echo faas-containerd gateway prometheus |xargs sudo ctr snapshot rm
echo faas-containerd gateway prometheus | xargs sudo ctr snapshot rm
```
## Links
@ -117,4 +174,3 @@ https://github.com/containernetworking/plugins
https://github.com/containerd/go-cni

View File

@ -2,10 +2,12 @@ package cmd
import (
"fmt"
"io"
"os"
"path"
systemd "github.com/alexellis/faasd/pkg/systemd"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
@ -16,24 +18,55 @@ var installCmd = &cobra.Command{
RunE: runInstall,
}
const faasdwd = "/run/faasd"
const faasContainerdwd = "/run/faas-containerd"
func runInstall(_ *cobra.Command, _ []string) error {
if err := ensureWorkingDir(path.Join(faasdwd, "secrets")); err != nil {
return err
}
if err := ensureWorkingDir(faasContainerdwd); err != nil {
return err
}
if basicAuthErr := makeBasicAuthFiles(path.Join(faasdwd, "secrets")); basicAuthErr != nil {
return errors.Wrap(basicAuthErr, "cannot create basic-auth-* files")
}
if err := cp("prometheus.yml", faasdwd); err != nil {
return err
}
if err := cp("resolv.conf", faasdwd); err != nil {
return err
}
err := binExists("/usr/local/bin/", "faas-containerd")
if err != nil {
return err
}
err = binExists("/usr/local/bin/", "faasd")
if err != nil {
return err
}
err = binExists("/usr/local/bin/", "netns")
if err != nil {
return err
}
err = systemd.InstallUnit("faas-containerd")
err = systemd.InstallUnit("faas-containerd", map[string]string{
"Cwd": faasContainerdwd,
"SecretMountPath": path.Join(faasdwd, "secrets")})
if err != nil {
return err
}
err = systemd.InstallUnit("faasd")
err = systemd.InstallUnit("faasd", map[string]string{"Cwd": faasdwd})
if err != nil {
return err
}
@ -73,3 +106,33 @@ func binExists(folder, name string) error {
}
return nil
}
func ensureWorkingDir(folder string) error {
if _, err := os.Stat(folder); err != nil {
err = os.MkdirAll(folder, 0600)
if err != nil {
return err
}
}
return nil
}
func cp(source, destFolder string) error {
file, err := os.Open(source)
if err != nil {
return err
}
defer file.Close()
out, err := os.Create(path.Join(destFolder, source))
if err != nil {
return err
}
defer out.Close()
_, err = io.Copy(out, file)
return err
}

View File

@ -3,19 +3,10 @@ package cmd
import (
"fmt"
"github.com/alexellis/faasd/pkg"
"github.com/morikuni/aec"
"github.com/spf13/cobra"
)
var (
// Version as per git repo
Version string
// GitCommit as per git repo
GitCommit string
)
// WelcomeMessage to introduce ofc-bootstrap
const WelcomeMessage = "Welcome to faasd"
@ -25,39 +16,14 @@ func init() {
rootCommand.AddCommand(installCmd)
}
var rootCommand = &cobra.Command{
Use: "faasd",
Short: "Start faasd",
Long: `
faasd - serverless without Kubernetes
`,
RunE: runRootCommand,
SilenceUsage: true,
}
var versionCmd = &cobra.Command{
Use: "version",
Short: "Display version information.",
Run: parseBaseCommand,
}
func getVersion() string {
if len(Version) != 0 {
return Version
}
return "dev"
}
func parseBaseCommand(_ *cobra.Command, _ []string) {
printLogo()
fmt.Printf(
`faasd
Commit: %s
Version: %s
`, pkg.GitCommit, pkg.GetVersion())
}
var (
// GitCommit Git Commit SHA
GitCommit string
// Version version of the CLI
Version string
)
// Execute faasd
func Execute(version, gitCommit string) error {
// Get Version and GitCommit values from main.go.
@ -70,6 +36,16 @@ func Execute(version, gitCommit string) error {
return nil
}
var rootCommand = &cobra.Command{
Use: "faasd",
Short: "Start faasd",
Long: `
faasd - serverless without Kubernetes
`,
RunE: runRootCommand,
SilenceUsage: true,
}
func runRootCommand(cmd *cobra.Command, args []string) error {
printLogo()
@ -78,7 +54,39 @@ func runRootCommand(cmd *cobra.Command, args []string) error {
return nil
}
var versionCmd = &cobra.Command{
Use: "version",
Short: "Display version information.",
Run: parseBaseCommand,
}
func parseBaseCommand(_ *cobra.Command, _ []string) {
printLogo()
fmt.Printf(
`faasd
Commit: %s
Version: %s
`, GitCommit, GetVersion())
}
func printLogo() {
logoText := aec.WhiteF.Apply(pkg.Logo)
logoText := aec.WhiteF.Apply(Logo)
fmt.Println(logoText)
}
// GetVersion get latest version
func GetVersion() string {
if len(Version) == 0 {
return "dev"
}
return Version
}
// Logo for version and root command
const Logo = ` __ _
/ _| __ _ __ _ ___ __| |
| |_ / _` + "`" + ` |/ _` + "`" + ` / __|/ _` + "`" + ` |
| _| (_| | (_| \__ \ (_| |
|_| \__,_|\__,_|___/\__,_|
`

164
cmd/up.go
View File

@ -2,15 +2,21 @@ package cmd
import (
"fmt"
"io/ioutil"
"log"
"os"
"os/signal"
"path"
"strings"
"sync"
"syscall"
"time"
"github.com/pkg/errors"
"github.com/alexellis/faasd/pkg"
"github.com/alexellis/k3sup/pkg/env"
"github.com/sethvargo/go-password/password"
"github.com/spf13/cobra"
)
@ -20,9 +26,34 @@ var upCmd = &cobra.Command{
RunE: runUp,
}
const secretMountDir = "/run/secrets"
func runUp(_ *cobra.Command, _ []string) error {
services := makeServiceDefinitions()
clientArch, clientOS := env.GetClientArch()
if clientOS != "Linux" {
return fmt.Errorf("You can only use faasd on Linux")
}
clientSuffix := ""
switch clientArch {
case "x86_64":
clientSuffix = ""
break
case "armhf":
case "armv7l":
clientSuffix = "-armhf"
break
case "arm64":
case "aarch64":
clientSuffix = "-arm64"
}
if basicAuthErr := makeBasicAuthFiles(path.Join(path.Join(faasdwd, "secrets"))); basicAuthErr != nil {
return errors.Wrap(basicAuthErr, "cannot create basic-auth-* files")
}
services := makeServiceDefinitions(clientSuffix)
start := time.Now()
supervisor, err := pkg.NewSupervisor("/run/containerd/containerd.sock")
@ -45,6 +76,8 @@ func runUp(_ *cobra.Command, _ []string) error {
log.Printf("Supervisor init done in: %s\n", time.Since(start).String())
shutdownTimeout := time.Second * 1
timeout := time.Second * 60
proxyDoneCh := make(chan bool)
wg := sync.WaitGroup{}
wg.Add(1)
@ -60,19 +93,107 @@ func runUp(_ *cobra.Command, _ []string) error {
if err != nil {
fmt.Println(err)
}
// Close proxy
proxyDoneCh <- true
time.AfterFunc(shutdownTimeout, func() {
wg.Done()
})
}()
gatewayURLChan := make(chan string, 1)
proxyPort := 8080
proxy := pkg.NewProxy(proxyPort, timeout)
go proxy.Start(gatewayURLChan, proxyDoneCh)
go func() {
wd, _ := os.Getwd()
time.Sleep(3 * time.Second)
fileData, fileErr := ioutil.ReadFile(path.Join(wd, "hosts"))
if fileErr != nil {
log.Println(fileErr)
return
}
host := ""
lines := strings.Split(string(fileData), "\n")
for _, line := range lines {
if strings.Index(line, "gateway") > -1 {
host = line[:strings.Index(line, "\t")]
}
}
log.Printf("[up] Sending %s to proxy\n", host)
gatewayURLChan <- host + ":8080"
close(gatewayURLChan)
}()
wg.Wait()
return nil
}
func makeServiceDefinitions() []pkg.Service {
func makeBasicAuthFiles(wd string) error {
pwdFile := wd + "/basic-auth-password"
authPassword, err := password.Generate(63, 10, 0, false, true)
if err != nil {
return err
}
err = makeFile(pwdFile, authPassword)
if err != nil {
return err
}
userFile := wd + "/basic-auth-user"
err = makeFile(userFile, "admin")
if err != nil {
return err
}
return nil
}
func makeFile(filePath, fileContents string) error {
_, err := os.Stat(filePath)
if err == nil {
log.Printf("File exists: %q\n", filePath)
return nil
} else if os.IsNotExist(err) {
log.Printf("Writing to: %q\n", filePath)
return ioutil.WriteFile(filePath, []byte(fileContents), 0644)
} else {
return err
}
}
func makeServiceDefinitions(archSuffix string) []pkg.Service {
wd, _ := os.Getwd()
return []pkg.Service{
pkg.Service{
Name: "basic-auth-plugin",
Image: "docker.io/openfaas/basic-auth-plugin:0.18.10" + archSuffix,
Env: []string{
"port=8080",
"secret_mount_path=" + secretMountDir,
"user_filename=basic-auth-user",
"pass_filename=basic-auth-password",
},
Mounts: []pkg.Mount{
pkg.Mount{
Src: path.Join(path.Join(wd, "secrets"), "basic-auth-password"),
Dest: path.Join(secretMountDir, "basic-auth-password"),
},
pkg.Mount{
Src: path.Join(path.Join(wd, "secrets"), "basic-auth-user"),
Dest: path.Join(secretMountDir, "basic-auth-user"),
},
},
Caps: []string{"CAP_NET_RAW"},
Args: nil,
},
pkg.Service{
Name: "nats",
Env: []string{""},
@ -95,7 +216,7 @@ func makeServiceDefinitions() []pkg.Service {
pkg.Service{
Name: "gateway",
Env: []string{
"basic_auth=false",
"basic_auth=true",
"functions_provider_url=http://faas-containerd:8081/",
"direct_functions=false",
"read_timeout=60s",
@ -103,10 +224,22 @@ func makeServiceDefinitions() []pkg.Service {
"upstream_timeout=65s",
"faas_nats_address=nats",
"faas_nats_port=4222",
"auth_proxy_url=http://basic-auth-plugin:8080/validate",
"auth_proxy_pass_body=false",
"secret_mount_path=" + secretMountDir,
},
Image: "docker.io/openfaas/gateway:0.18.7",
Mounts: []pkg.Mount{},
Caps: []string{"CAP_NET_RAW"},
Image: "docker.io/openfaas/gateway:0.18.8" + archSuffix,
Mounts: []pkg.Mount{
pkg.Mount{
Src: path.Join(path.Join(wd, "secrets"), "basic-auth-password"),
Dest: path.Join(secretMountDir, "basic-auth-password"),
},
pkg.Mount{
Src: path.Join(path.Join(wd, "secrets"), "basic-auth-user"),
Dest: path.Join(secretMountDir, "basic-auth-user"),
},
},
Caps: []string{"CAP_NET_RAW"},
},
pkg.Service{
Name: "queue-worker",
@ -117,11 +250,22 @@ func makeServiceDefinitions() []pkg.Service {
"faas_gateway_address=gateway",
"ack_wait=5m5s",
"max_inflight=1",
"faas_print_body=true",
"write_debug=false",
"basic_auth=true",
"secret_mount_path=" + secretMountDir,
},
Image: "docker.io/openfaas/queue-worker:0.9.0",
Mounts: []pkg.Mount{},
Caps: []string{"CAP_NET_RAW"},
Image: "docker.io/openfaas/queue-worker:0.9.0",
Mounts: []pkg.Mount{
pkg.Mount{
Src: path.Join(path.Join(wd, "secrets"), "basic-auth-password"),
Dest: path.Join(secretMountDir, "basic-auth-password"),
},
pkg.Mount{
Src: path.Join(path.Join(wd, "secrets"), "basic-auth-user"),
Dest: path.Join(secretMountDir, "basic-auth-user"),
},
},
Caps: []string{"CAP_NET_RAW"},
},
}
}

29
hack/build-containerd-armhf.sh Executable file
View File

@ -0,0 +1,29 @@
#!/bin/bash
export ARCH="armv6l"
echo "Downloading Go"
curl -SLsf https://dl.google.com/go/go1.12.14.linux-$ARCH.tar.gz --output /tmp/go.tgz
sudo rm -rf /usr/local/go/
sudo mkdir -p /usr/local/go/
sudo tar -xvf /tmp/go.tgz -C /usr/local/go/ --strip-components=1
export GOPATH=$HOME/go/
export PATH=$PATH:/usr/local/go/bin/
go version
echo "Building containerd"
mkdir -p $GOPATH/src/github.com/containerd
cd $GOPATH/src/github.com/containerd
git clone https://github.com/containerd/containerd
cd containerd
git fetch origin --tags
git checkout v1.3.2
make
sudo make install
sudo containerd --version

29
hack/build-containerd.sh Normal file
View File

@ -0,0 +1,29 @@
#!/bin/bash
export ARCH="amd64"
echo "Downloading Go"
curl -SLsf https://dl.google.com/go/go1.12.14.linux-$ARCH.tar.gz --output /tmp/go.tgz
sudo rm -rf /usr/local/go/
sudo mkdir -p /usr/local/go/
sudo tar -xvf /tmp/go.tgz -C /usr/local/go/ --strip-components=1
export GOPATH=$HOME/go/
export PATH=$PATH:/usr/local/go/bin/
go version
echo "Building containerd"
mkdir -p $GOPATH/src/github.com/containerd
cd $GOPATH/src/github.com/containerd
git clone https://github.com/containerd/containerd
cd containerd
git fetch origin --tags
git checkout v1.3.2
make
sudo make install
sudo containerd --version

View File

@ -3,10 +3,11 @@ Description=faasd-containerd
[Service]
MemoryLimit=500M
Environment="secret_mount_path={{.SecretMountPath}}"
ExecStart=/usr/local/bin/faas-containerd
Restart=on-failure
RestartSec=10s
WorkingDirectory=/usr/local/bin/
WorkingDirectory={{.Cwd}}
[Install]
WantedBy=multi-user.target

View File

@ -4,7 +4,7 @@ After=faas-containerd.service
[Service]
MemoryLimit=500M
ExecStart={{.Cwd}}/faasd up
ExecStart=/usr/local/bin/faasd up
Restart=on-failure
RestartSec=10s
WorkingDirectory={{.Cwd}}

12
main.go
View File

@ -4,13 +4,19 @@ import (
"os"
"github.com/alexellis/faasd/cmd"
"github.com/alexellis/faasd/pkg"
)
// These values will be injected into these variables at the build time.
var (
// GitCommit Git Commit SHA
GitCommit string
// Version version of the CLI
Version string
)
func main() {
if err := cmd.Execute(pkg.Version, pkg.GitCommit); err != nil {
if err := cmd.Execute(Version, GitCommit); err != nil {
os.Exit(1)
}
return
}

122
pkg/proxy.go Normal file
View File

@ -0,0 +1,122 @@
package pkg
import (
"context"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"time"
)
func NewProxy(port int, timeout time.Duration) *Proxy {
return &Proxy{
Port: port,
Timeout: timeout,
}
}
type Proxy struct {
Timeout time.Duration
Port int
}
func (p *Proxy) Start(gatewayChan chan string, done chan bool) error {
tcp := p.Port
http.DefaultClient.CheckRedirect = func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
}
ps := proxyState{
Host: "",
}
ps.Host = <-gatewayChan
log.Printf("Starting faasd proxy on %d\n", tcp)
fmt.Printf("Gateway: %s\n", ps.Host)
s := &http.Server{
Addr: fmt.Sprintf(":%d", tcp),
ReadTimeout: p.Timeout,
WriteTimeout: p.Timeout,
MaxHeaderBytes: 1 << 20, // Max header of 1MB
Handler: http.HandlerFunc(makeProxy(&ps)),
}
go func() {
log.Printf("[proxy] Begin listen on %d\n", p.Port)
if err := s.ListenAndServe(); err != http.ErrServerClosed {
log.Printf("Error ListenAndServe: %v", err)
}
}()
log.Println("[proxy] Wait for done")
<-done
log.Println("[proxy] Done received")
if err := s.Shutdown(context.Background()); err != nil {
log.Printf("[proxy] Error in Shutdown: %v", err)
}
return nil
}
// copyHeaders clones the header values from the source into the destination.
func copyHeaders(destination http.Header, source *http.Header) {
for k, v := range *source {
vClone := make([]string, len(v))
copy(vClone, v)
destination[k] = vClone
}
}
type proxyState struct {
Host string
}
func makeProxy(ps *proxyState) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
query := ""
if len(r.URL.RawQuery) > 0 {
query = "?" + r.URL.RawQuery
}
upstream := fmt.Sprintf("http://%s%s%s", ps.Host, r.URL.Path, query)
fmt.Printf("[faasd] proxy: %s\n", upstream)
if r.Body != nil {
defer r.Body.Close()
}
wrapper := ioutil.NopCloser(r.Body)
upReq, upErr := http.NewRequest(r.Method, upstream, wrapper)
copyHeaders(upReq.Header, &r.Header)
if upErr != nil {
log.Println(upErr)
http.Error(w, upErr.Error(), http.StatusInternalServerError)
return
}
upRes, upResErr := http.DefaultClient.Do(upReq)
if upResErr != nil {
log.Println(upResErr)
http.Error(w, upResErr.Error(), http.StatusInternalServerError)
return
}
copyHeaders(w.Header(), &upRes.Header)
w.WriteHeader(upRes.StatusCode)
io.Copy(w, upRes.Body)
}
}

73
pkg/proxy_test.go Normal file
View File

@ -0,0 +1,73 @@
package pkg
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"net/http/httptest"
"net/url"
"sync"
"testing"
"time"
)
func Test_Proxy_ToPrivateServer(t *testing.T) {
wantBodyText := "OK"
wantBody := []byte(wantBodyText)
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Body != nil {
defer r.Body.Close()
}
w.WriteHeader(http.StatusOK)
w.Write(wantBody)
}))
defer upstream.Close()
port := 8080
proxy := NewProxy(port, time.Second*1)
gwChan := make(chan string, 1)
doneCh := make(chan bool)
go proxy.Start(gwChan, doneCh)
u, _ := url.Parse(upstream.URL)
log.Println("Host", u.Host)
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
gwChan <- u.Host
wg.Done()
}()
wg.Wait()
req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("http://127.0.0.1:%d", port), nil)
if err != nil {
t.Fatal(err)
}
for i := 1; i < 11; i++ {
res, err := http.DefaultClient.Do(req)
if err != nil {
t.Logf("Try %d, gave error: %s", i, err)
time.Sleep(time.Millisecond * 100)
} else {
resBody, _ := ioutil.ReadAll(res.Body)
if string(resBody) != string(wantBody) {
t.Errorf("want %s, but got %s in body", string(wantBody), string(resBody))
}
break
}
}
go func() {
doneCh <- true
}()
}

117
pkg/service/service.go Normal file
View File

@ -0,0 +1,117 @@
package service
import (
"context"
"fmt"
"log"
"sync"
"time"
"github.com/containerd/containerd"
"github.com/containerd/containerd/errdefs"
"golang.org/x/sys/unix"
)
// Remove removes a container
func Remove(ctx context.Context, client *containerd.Client, name string) error {
container, containerErr := client.LoadContainer(ctx, name)
if containerErr == nil {
found := true
t, err := container.Task(ctx, nil)
if err != nil {
if errdefs.IsNotFound(err) {
found = false
} else {
return fmt.Errorf("unable to get task %s: ", err)
}
}
if found {
status, _ := t.Status(ctx)
fmt.Printf("Status of %s is: %s\n", name, status.Status)
log.Printf("Need to kill %s\n", name)
err := killTask(ctx, t)
if err != nil {
return fmt.Errorf("error killing task %s, %s, %s", container.ID(), name, err)
}
}
err = container.Delete(ctx, containerd.WithSnapshotCleanup)
if err != nil {
return fmt.Errorf("error deleting container %s, %s, %s", container.ID(), name, err)
}
} else {
service := client.SnapshotService("")
key := name + "snapshot"
if _, err := client.SnapshotService("").Stat(ctx, key); err == nil {
service.Remove(ctx, key)
}
}
return nil
}
// From Stellar
func killTask(ctx context.Context, task containerd.Task) error {
wg := &sync.WaitGroup{}
wg.Add(1)
var err error
go func() {
defer wg.Done()
if task != nil {
wait, err := task.Wait(ctx)
if err != nil {
err = fmt.Errorf("error waiting on task: %s", err)
return
}
if err := task.Kill(ctx, unix.SIGTERM, containerd.WithKillAll); err != nil {
log.Printf("error killing container task: %s", err)
}
select {
case <-wait:
task.Delete(ctx)
return
case <-time.After(5 * time.Second):
if err := task.Kill(ctx, unix.SIGKILL, containerd.WithKillAll); err != nil {
log.Printf("error force killing container task: %s", err)
}
return
}
}
}()
wg.Wait()
return err
}
func PrepareImage(ctx context.Context, client *containerd.Client, imageName, snapshotter string) (containerd.Image, error) {
var empty containerd.Image
image, err := client.GetImage(ctx, imageName)
if err != nil {
if !errdefs.IsNotFound(err) {
return empty, err
}
img, err := client.Pull(ctx, imageName, containerd.WithPullUnpack)
if err != nil {
return empty, fmt.Errorf("cannot pull: %s", err)
}
image = img
}
unpacked, err := image.IsUnpacked(ctx, snapshotter)
if err != nil {
return empty, fmt.Errorf("cannot check if unpacked: %s", err)
}
if !unpacked {
if err := image.Unpack(ctx, snapshotter); err != nil {
return empty, fmt.Errorf("cannot unpack: %s", err)
}
}
return image, nil
}

View File

@ -8,15 +8,12 @@ import (
"os"
"os/exec"
"path"
"sync"
"time"
"github.com/alexellis/faasd/pkg/service"
"github.com/alexellis/faasd/pkg/weave"
"github.com/containerd/containerd"
"github.com/containerd/containerd/cio"
"github.com/containerd/containerd/containers"
"github.com/containerd/containerd/errdefs"
"golang.org/x/sys/unix"
"github.com/containerd/containerd/namespaces"
"github.com/containerd/containerd/oci"
@ -48,7 +45,7 @@ func (s *Supervisor) Remove(svcs []Service) error {
ctx := namespaces.WithNamespace(context.Background(), "default")
for _, svc := range svcs {
err := removeContainer(ctx, s.client, svc.Name)
err := service.Remove(ctx, s.client, svc.Name)
if err != nil {
return err
}
@ -75,7 +72,7 @@ func (s *Supervisor) Start(svcs []Service) error {
for _, svc := range svcs {
fmt.Printf("Preparing: %s with image: %s\n", svc.Name, svc.Image)
img, err := prepareImage(ctx, s.client, svc.Image)
img, err := service.PrepareImage(ctx, s.client, svc.Image, defaultSnapshotter)
if err != nil {
return err
}
@ -87,13 +84,13 @@ func (s *Supervisor) Start(svcs []Service) error {
for _, svc := range svcs {
fmt.Printf("Reconciling: %s\n", svc.Name)
image := images[svc.Name]
containerErr := removeContainer(ctx, s.client, svc.Name)
containerErr := service.Remove(ctx, s.client, svc.Name)
if containerErr != nil {
return containerErr
}
image := images[svc.Name]
mounts := []specs.Mount{}
if len(svc.Mounts) > 0 {
for _, mnt := range svc.Mounts {
@ -156,7 +153,7 @@ func (s *Supervisor) Start(svcs []Service) error {
)
if containerCreateErr != nil {
log.Println(containerCreateErr)
log.Printf("Error creating container %s\n", containerCreateErr)
return containerCreateErr
}
@ -164,7 +161,7 @@ func (s *Supervisor) Start(svcs []Service) error {
task, err := newContainer.NewTask(ctx, cio.NewCreator(cio.WithStdio))
if err != nil {
log.Println(err)
log.Printf("Error creating task: %s\n", err)
return err
}
@ -178,20 +175,21 @@ func (s *Supervisor) Start(svcs []Service) error {
writeErr := ioutil.WriteFile("hosts", hosts, 0644)
if writeErr != nil {
log.Println("Error writing hosts file")
log.Printf("Error writing file %s %s\n", "hosts", writeErr)
}
// os.Chown("hosts", 101, 101)
exitStatusC, err := task.Wait(ctx)
_, err = task.Wait(ctx)
if err != nil {
log.Println(err)
log.Printf("Wait err: %s\n", err)
return err
}
log.Println("Exited: ", exitStatusC)
// call start on the task to execute the redis server
if err := task.Start(ctx); err != nil {
log.Println("Task err: ", err)
log.Printf("Task: %s\tContainer: %s\n", task.ID(), newContainer.ID())
// log.Println("Exited: ", exitStatusC)
if err = task.Start(ctx); err != nil {
log.Printf("Task err: %s\n", err)
return err
}
}
@ -199,37 +197,6 @@ func (s *Supervisor) Start(svcs []Service) error {
return nil
}
func prepareImage(ctx context.Context, client *containerd.Client, imageName string) (containerd.Image, error) {
snapshotter := defaultSnapshotter
var empty containerd.Image
image, err := client.GetImage(ctx, imageName)
if err != nil {
if !errdefs.IsNotFound(err) {
return empty, err
}
img, err := client.Pull(ctx, imageName, containerd.WithPullUnpack)
if err != nil {
return empty, fmt.Errorf("cannot pull: %s", err)
}
image = img
}
unpacked, err := image.IsUnpacked(ctx, snapshotter)
if err != nil {
return empty, fmt.Errorf("cannot check if unpacked: %s", err)
}
if !unpacked {
if err := image.Unpack(ctx, snapshotter); err != nil {
return empty, fmt.Errorf("cannot unpack: %s", err)
}
}
return image, nil
}
func getIP(containerID string, taskPID uint32) string {
// https://github.com/weaveworks/weave/blob/master/net/netdev.go
@ -274,70 +241,3 @@ func withOCIArgs(args []string) oci.SpecOpts {
}
}
// From Stellar
func killTask(ctx context.Context, task containerd.Task) error {
wg := &sync.WaitGroup{}
wg.Add(1)
var err error
go func() {
defer wg.Done()
if task != nil {
wait, err := task.Wait(ctx)
if err != nil {
err = fmt.Errorf("error waiting on task: %s", err)
return
}
if err := task.Kill(ctx, unix.SIGTERM, containerd.WithKillAll); err != nil {
log.Printf("error killing container task: %s", err)
}
select {
case <-wait:
task.Delete(ctx)
return
case <-time.After(5 * time.Second):
if err := task.Kill(ctx, unix.SIGKILL, containerd.WithKillAll); err != nil {
log.Printf("error force killing container task: %s", err)
}
return
}
}
}()
wg.Wait()
return err
}
func removeContainer(ctx context.Context, client *containerd.Client, name string) error {
container, containerErr := client.LoadContainer(ctx, name)
if containerErr == nil {
found := true
t, err := container.Task(ctx, nil)
if err != nil {
if errdefs.IsNotFound(err) {
found = false
} else {
return fmt.Errorf("unable to get task %s: ", err)
}
}
if found {
status, _ := t.Status(ctx)
fmt.Printf("Status of %s is: %s\n", name, status.Status)
log.Printf("Need to kill %s\n", name)
err := killTask(ctx, t)
if err != nil {
return fmt.Errorf("error killing task %s, %s, %s", container.ID(), name, err)
}
}
err = container.Delete(ctx, containerd.WithSnapshotCleanup)
if err != nil {
return fmt.Errorf("error deleting container %s, %s, %s", container.ID(), name, err)
}
}
return nil
}

View File

@ -64,19 +64,21 @@ func DaemonReload() error {
return nil
}
func InstallUnit(name string) error {
tmpl, err := template.ParseFiles("./hack/" + name + ".service")
wd, _ := os.Getwd()
var tpl bytes.Buffer
userData := struct {
Cwd string
}{
Cwd: wd,
func InstallUnit(name string, tokens map[string]string) error {
if len(tokens["Cwd"]) == 0 {
return fmt.Errorf("key Cwd expected in tokens parameter")
}
err = tmpl.Execute(&tpl, userData)
tmplName := "./hack/" + name + ".service"
tmpl, err := template.ParseFiles(tmplName)
if err != nil {
return fmt.Errorf("error loading template %s, error %s", tmplName, err)
}
var tpl bytes.Buffer
err = tmpl.Execute(&tpl, tokens)
if err != nil {
return err
}

View File

@ -1,23 +1 @@
package pkg
var (
//GitCommit Git Commit SHA
GitCommit string
//Version version of the CLI
Version string
)
//GetVersion get latest version
func GetVersion() string {
if len(Version) == 0 {
return "dev"
}
return Version
}
const Logo = ` __ _
/ _| __ _ __ _ ___ __| |
| |_ / _` + "`" + ` |/ _` + "`" + ` / __|/ _` + "`" + ` |
| _| (_| | (_| \__ \ (_| |
|_| \__,_|\__,_|___/\__,_|
`

21
vendor/github.com/alexellis/go-execute/LICENSE generated vendored Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2019 Inlets
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

117
vendor/github.com/alexellis/go-execute/pkg/v1/exec.go generated vendored Normal file
View File

@ -0,0 +1,117 @@
package execute
import (
"bytes"
"fmt"
"io"
"os"
"os/exec"
"strings"
)
type ExecTask struct {
Command string
Args []string
Shell bool
Env []string
Cwd string
// StreamStdio prints stdout and stderr directly to os.Stdout/err as
// the command runs.
StreamStdio bool
// PrintCommand prints the command before executing
PrintCommand bool
}
type ExecResult struct {
Stdout string
Stderr string
ExitCode int
}
func (et ExecTask) Execute() (ExecResult, error) {
argsSt := ""
if len(et.Args) > 0 {
argsSt = strings.Join(et.Args, " ")
}
if et.PrintCommand {
fmt.Println("exec: ", et.Command, argsSt)
}
var cmd *exec.Cmd
if et.Shell {
var args []string
if len(et.Args) == 0 {
startArgs := strings.Split(et.Command, " ")
script := strings.Join(startArgs, " ")
args = append([]string{"-c"}, fmt.Sprintf("%s", script))
} else {
script := strings.Join(et.Args, " ")
args = append([]string{"-c"}, fmt.Sprintf("%s %s", et.Command, script))
}
cmd = exec.Command("/bin/bash", args...)
} else {
if strings.Index(et.Command, " ") > 0 {
parts := strings.Split(et.Command, " ")
command := parts[0]
args := parts[1:]
cmd = exec.Command(command, args...)
} else {
cmd = exec.Command(et.Command, et.Args...)
}
}
cmd.Dir = et.Cwd
if len(et.Env) > 0 {
cmd.Env = os.Environ()
for _, env := range et.Env {
cmd.Env = append(cmd.Env, env)
}
}
stdoutBuff := bytes.Buffer{}
stderrBuff := bytes.Buffer{}
var stdoutWriters io.Writer
var stderrWriters io.Writer
if et.StreamStdio {
stdoutWriters = io.MultiWriter(os.Stdout, &stdoutBuff)
stderrWriters = io.MultiWriter(os.Stderr, &stderrBuff)
} else {
stdoutWriters = &stdoutBuff
stderrWriters = &stderrBuff
}
cmd.Stdout = stdoutWriters
cmd.Stderr = stderrWriters
startErr := cmd.Start()
if startErr != nil {
return ExecResult{}, startErr
}
exitCode := 0
execErr := cmd.Wait()
if execErr != nil {
if exitError, ok := execErr.(*exec.ExitError); ok {
exitCode = exitError.ExitCode()
}
}
return ExecResult{
Stdout: string(stdoutBuff.Bytes()),
Stderr: string(stderrBuff.Bytes()),
ExitCode: exitCode,
}, nil
}

21
vendor/github.com/alexellis/k3sup/LICENSE generated vendored Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2019 Alex Ellis
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

29
vendor/github.com/alexellis/k3sup/pkg/env/env.go generated vendored Normal file
View File

@ -0,0 +1,29 @@
package env
import (
"log"
"strings"
execute "github.com/alexellis/go-execute/pkg/v1"
)
// GetClientArch returns a pair of arch and os
func GetClientArch() (string, string) {
task := execute.ExecTask{Command: "uname", Args: []string{"-m"}}
res, err := task.Execute()
if err != nil {
log.Println(err)
}
arch := strings.TrimSpace(res.Stdout)
taskOS := execute.ExecTask{Command: "uname", Args: []string{"-s"}}
resOS, errOS := taskOS.Execute()
if errOS != nil {
log.Println(errOS)
}
os := strings.TrimSpace(resOS.Stdout)
return arch, os
}

20
vendor/github.com/sethvargo/go-password/LICENSE generated vendored Normal file
View File

@ -0,0 +1,20 @@
Copyright 2017 Seth Vargo <seth@sethvargo.com>
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@ -0,0 +1,253 @@
// Package password provides a library for generating high-entropy random
// password strings via the crypto/rand package.
//
// res, err := Generate(64, 10, 10, false, false)
// if err != nil {
// log.Fatal(err)
// }
// log.Printf(res)
//
// Most functions are safe for concurrent use.
package password
import (
"crypto/rand"
"errors"
"math/big"
"strings"
)
// Built-time checks that the generators implement the interface.
var _ PasswordGenerator = (*Generator)(nil)
// PasswordGenerator is an interface that implements the Generate function. This
// is useful for testing where you can pass this interface instead of a real
// password generator to mock responses for predicability.
type PasswordGenerator interface {
Generate(int, int, int, bool, bool) (string, error)
MustGenerate(int, int, int, bool, bool) string
}
const (
// LowerLetters is the list of lowercase letters.
LowerLetters = "abcdefghijklmnopqrstuvwxyz"
// UpperLetters is the list of uppercase letters.
UpperLetters = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
// Digits is the list of permitted digits.
Digits = "0123456789"
// Symbols is the list of symbols.
Symbols = "~!@#$%^&*()_+`-={}|[]\\:\"<>?,./"
)
var (
// ErrExceedsTotalLength is the error returned with the number of digits and
// symbols is greater than the total length.
ErrExceedsTotalLength = errors.New("number of digits and symbols must be less than total length")
// ErrLettersExceedsAvailable is the error returned with the number of letters
// exceeds the number of available letters and repeats are not allowed.
ErrLettersExceedsAvailable = errors.New("number of letters exceeds available letters and repeats are not allowed")
// ErrDigitsExceedsAvailable is the error returned with the number of digits
// exceeds the number of available digits and repeats are not allowed.
ErrDigitsExceedsAvailable = errors.New("number of digits exceeds available digits and repeats are not allowed")
// ErrSymbolsExceedsAvailable is the error returned with the number of symbols
// exceeds the number of available symbols and repeats are not allowed.
ErrSymbolsExceedsAvailable = errors.New("number of symbols exceeds available symbols and repeats are not allowed")
)
// Generator is the stateful generator which can be used to customize the list
// of letters, digits, and/or symbols.
type Generator struct {
lowerLetters string
upperLetters string
digits string
symbols string
}
// GeneratorInput is used as input to the NewGenerator function.
type GeneratorInput struct {
LowerLetters string
UpperLetters string
Digits string
Symbols string
}
// NewGenerator creates a new Generator from the specified configuration. If no
// input is given, all the default values are used. This function is safe for
// concurrent use.
func NewGenerator(i *GeneratorInput) (*Generator, error) {
if i == nil {
i = new(GeneratorInput)
}
g := &Generator{
lowerLetters: i.LowerLetters,
upperLetters: i.UpperLetters,
digits: i.Digits,
symbols: i.Symbols,
}
if g.lowerLetters == "" {
g.lowerLetters = LowerLetters
}
if g.upperLetters == "" {
g.upperLetters = UpperLetters
}
if g.digits == "" {
g.digits = Digits
}
if g.symbols == "" {
g.symbols = Symbols
}
return g, nil
}
// Generate generates a password with the given requirements. length is the
// total number of characters in the password. numDigits is the number of digits
// to include in the result. numSymbols is the number of symbols to include in
// the result. noUpper excludes uppercase letters from the results. allowRepeat
// allows characters to repeat.
//
// The algorithm is fast, but it's not designed to be performant; it favors
// entropy over speed. This function is safe for concurrent use.
func (g *Generator) Generate(length, numDigits, numSymbols int, noUpper, allowRepeat bool) (string, error) {
letters := g.lowerLetters
if !noUpper {
letters += g.upperLetters
}
chars := length - numDigits - numSymbols
if chars < 0 {
return "", ErrExceedsTotalLength
}
if !allowRepeat && chars > len(letters) {
return "", ErrLettersExceedsAvailable
}
if !allowRepeat && numDigits > len(g.digits) {
return "", ErrDigitsExceedsAvailable
}
if !allowRepeat && numSymbols > len(g.symbols) {
return "", ErrSymbolsExceedsAvailable
}
var result string
// Characters
for i := 0; i < chars; i++ {
ch, err := randomElement(letters)
if err != nil {
return "", err
}
if !allowRepeat && strings.Contains(result, ch) {
i--
continue
}
result, err = randomInsert(result, ch)
if err != nil {
return "", err
}
}
// Digits
for i := 0; i < numDigits; i++ {
d, err := randomElement(g.digits)
if err != nil {
return "", err
}
if !allowRepeat && strings.Contains(result, d) {
i--
continue
}
result, err = randomInsert(result, d)
if err != nil {
return "", err
}
}
// Symbols
for i := 0; i < numSymbols; i++ {
sym, err := randomElement(g.symbols)
if err != nil {
return "", err
}
if !allowRepeat && strings.Contains(result, sym) {
i--
continue
}
result, err = randomInsert(result, sym)
if err != nil {
return "", err
}
}
return result, nil
}
// MustGenerate is the same as Generate, but panics on error.
func (g *Generator) MustGenerate(length, numDigits, numSymbols int, noUpper, allowRepeat bool) string {
res, err := g.Generate(length, numDigits, numSymbols, noUpper, allowRepeat)
if err != nil {
panic(err)
}
return res
}
// Generate is the package shortcut for Generator.Generate.
func Generate(length, numDigits, numSymbols int, noUpper, allowRepeat bool) (string, error) {
gen, err := NewGenerator(nil)
if err != nil {
return "", err
}
return gen.Generate(length, numDigits, numSymbols, noUpper, allowRepeat)
}
// MustGenerate is the package shortcut for Generator.MustGenerate.
func MustGenerate(length, numDigits, numSymbols int, noUpper, allowRepeat bool) string {
res, err := Generate(length, numDigits, numSymbols, noUpper, allowRepeat)
if err != nil {
panic(err)
}
return res
}
// randomInsert randomly inserts the given value into the given string.
func randomInsert(s, val string) (string, error) {
if s == "" {
return val, nil
}
n, err := rand.Int(rand.Reader, big.NewInt(int64(len(s)+1)))
if err != nil {
return "", err
}
i := n.Int64()
return s[0:i] + val + s[i:], nil
}
// randomElement extracts a random element from the given string.
func randomElement(s string) (string, error) {
n, err := rand.Int(rand.Reader, big.NewInt(int64(len(s))))
if err != nil {
return "", err
}
return string(s[n.Int64()]), nil
}

View File

@ -0,0 +1,39 @@
package password
// Built-time checks that the generators implement the interface.
var _ PasswordGenerator = (*mockGenerator)(nil)
type mockGenerator struct {
result string
err error
}
// NewMockGenerator creates a new generator that satisfies the PasswordGenerator
// interface. If an error is provided, the error is returned. If a result if
// provided, the result is always returned, regardless of what parameters are
// passed into the Generate or MustGenerate methods.
//
// This function is most useful for tests where you want to have predicable
// results for a transitive resource that depends on go-password.
func NewMockGenerator(result string, err error) *mockGenerator {
return &mockGenerator{
result: result,
err: err,
}
}
// Generate returns the mocked result or error.
func (g *mockGenerator) Generate(int, int, int, bool, bool) (string, error) {
if g.err != nil {
return "", g.err
}
return g.result, nil
}
// MustGenerate returns the mocked result or panics if an error was given.
func (g *mockGenerator) MustGenerate(int, int, int, bool, bool) string {
if g.err != nil {
panic(g.err)
}
return g.result
}