Compare commits

...

31 Commits

Author SHA1 Message Date
77867f17e3 Migrate to Go 1.17
Tested during local development and deployment to multipass
and Ubuntu. Worked as expected.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-12-21 13:26:38 +00:00
5aed707354 Create volumes automatically for NATS/Prometheus
Fixes: #223

Tested by checking the logs of Prometheus and NATS was tested
by running async requests with hey then restarting faasd,
which deletes the NATS and queue-worker containers. The work
left over in the queue was restarted as expected.

Pre-created volumes are read from docker-compose.yaml and
only numeric user IDs are supported at this time. Users
can still specify a textual username, if they create the
source directory for a mount before restarting faasd,
it won't try to overwrite the directory.

Add comment for workingDirectoryPermission

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-12-21 13:26:38 +00:00
8fbdd1a461 Update ROADMAP.md 2021-11-14 08:58:52 +00:00
8dd48b8957 Update README.md 2021-11-14 08:56:56 +00:00
6763ed6d66 Update README.md 2021-11-14 08:55:53 +00:00
acb5d0bd1c Amend patches.md gh command to include 'pr'
Signed-off-by: Richard Gee richard@technologee.co.uk
2021-11-03 19:48:39 +00:00
2c9eb3904e Add guide for testing patches.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-11-01 13:00:26 +00:00
b42066d1a1 Fixed bad memory display and refactor test cases in functions_test.go
Signed-off-by: Shikachuu <zcmate@gmail.com>
2021-11-01 10:07:25 +00:00
17188b8de9 Added unit tests for readMemoryLimitFromSpec
Signed-off-by: Shikachuu <zcmate@gmail.com>
2021-11-01 10:07:25 +00:00
0c0088e8b0 Change readMemoryLimitFromSpec, to a more clear implementation, edited error message.
Signed-off-by: Shikachuu <zcmate@gmail.com>
2021-11-01 10:07:25 +00:00
c5f167df21 Change plain number response to Decimal String.
Signed-off-by: Shikachuu <zcmate@gmail.com>
2021-11-01 10:07:25 +00:00
d5fcc7b2ab Fixed nil pointer dereference while parsing memory limit
Signed-off-by: Shikachuu <zcmate@gmail.com>
2021-11-01 10:07:25 +00:00
cbfefb6fa5 Extend the Function type with a memoryLimit field, create a conversion to k8s resource value and return it through the REST API.
Signed-off-by: Shikachuu <zcmate@gmail.com>
2021-11-01 10:07:25 +00:00
ea62c1b12d feat: add support for raw secret values
Load the secret value from the RawValue field, if it is empty, use the
string value. Add unit tests for the creation handler.

Refactor secret parser tests.

Resolves #208

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2021-10-17 18:04:06 +01:00
8f40618a5c Update README.md 2021-10-17 15:49:25 +01:00
3fe0d8d8d3 Update messages to want/got for unit tests
This is the style used in the openfaas project.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-09-16 10:44:36 +01:00
5aa4c69e03 Inline namespace check and create const for label
* Inlines the namespace check for valid faasd namespaces
* Creates a const for the namespace label applied to faasd
namespaces

Tested with go build and go test.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-09-16 10:43:21 +01:00
12b5e8ca7f Add check for namespace label openfaas=true
This commit adds the checks that the namespace supplied by the user has
the `openfaas=true` label. Without this check the user can
deploy/update/read functions in any namespace  using the CLI.

The UI is not effected because it calls the listnamesaces endpoint,
which has the check for the label

Signed-off-by: Alistair Hey <alistair@heyal.co.uk>
2021-09-16 10:37:32 +01:00
195e81f595 Fix for #201
Old secrets are now copied, rather than moved, so that any
existing functions do not need to be redeployed by the user.

As a maintenance task, users should remove the older secrets.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-09-15 19:49:28 +01:00
06fbca83bf Fix syntax error with error wrapping
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-09-15 16:17:12 +01:00
e71d2c27c5 Update some errors to wrapped syntax
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-09-15 15:54:44 +01:00
13f4a487ce Correct error formatting
Errors should not start with an uppercase letter.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-09-15 15:52:13 +01:00
13412841aa Rename getMounts to getOSMounts
A more descriptive name

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-09-15 15:51:43 +01:00
e76d0d34ba Add pacman to the install script
The install.sh script was modified to include a test for the pacman package manager, and to use it should it be present.
This is necessary for the script to work on Arch based Linux distributions, or more generally ones that use pacman as their main package manager.
It was tested by simply trying it out on two local machines, one running Manjaro, one running Arch. In both cases it worked as expected, and without error.

Signed-off-by: Jacob Palecek <jacob.palecek@outlook.com>
2021-09-15 12:50:04 +01:00
dec02f3240 Enable multi namespace support
Signed-off-by: Nitishkumar Singh <nitishkumarsingh71@gmail.com>

Included Test cases for utils

Signed-off-by: Nitishkumar Singh <nitishkumarsingh71@gmail.com>

Multi namespace handling in invoke

Signed-off-by: Nitishkumar Singh <nitishkumarsingh71@gmail.com>

List Namespaces capability included

Signed-off-by: Nitishkumar Singh <nitishkumarsingh71@gmail.com>

remove faasd namespace from list result

Signed-off-by: Nitishkumar Singh <nitishkumarsingh71@gmail.com>

Create Secret Folder Path

Signed-off-by: Nitishkumar Singh <nitishkumarsingh71@gmail.com>

Filter only namespaces with openfass label

Signed-off-by: Nitishkumar Singh <nitishkumarsingh71@gmail.com>

Include Testcase for utility function

Signed-off-by: Nitishkumar Singh <nitishkumarsingh71@gmail.com>

move default function secets to openfaas-fn namespace secrets

Signed-off-by: Nitishkumar Singh <nitishkumarsingh71@gmail.com>

Corrected issue with secret moving

Signed-off-by: Nitishkumar Singh <nitishkumarsingh71@gmail.com>
2021-09-15 12:47:52 +01:00
73c7349e36 Refactor hosts_dir lookup
Applies feedback from #199 to inline the hosts_dir env-var
lookup.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-09-15 12:44:56 +01:00
b8ada0d46b Changed default and fallback host_dir
Have to change them since we are trying to resolve system services and the system services are in the /var/lib/faasd/hosts file instead of /var/lib/faasd-provider/hosts file.

Signed-off-by: Shikachuu <zcmate@gmail.com>
2021-09-15 12:42:02 +01:00
5ac51663da Added default value in case of missing env-var
Signed-off-by: Shikachuu <zcmate@gmail.com>
2021-09-15 12:42:02 +01:00
1e9d8fffa0 Updated the env-var usage, as requested from the review of alexellis on the previous PR by utsavanand2
https://github.com/openfaas/faasd/pull/154#discussion_r608777877

Signed-off-by: Shikachuu <zcmate@gmail.com>
2021-09-15 12:42:02 +01:00
57322c4947 Update terraform scripts to latest version
Signed-off-by: Engin Diri <engin.diri@mail.schwarz>
2021-08-25 17:04:56 +01:00
6b840f0226 Upgrade scripts for faasd 0.13.0
Upgrade to 0.13.0 and add build script for containerd on arm64

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2021-07-27 10:36:22 +01:00
42 changed files with 1163 additions and 220 deletions

View File

@ -12,7 +12,7 @@ jobs:
GO111MODULE: off GO111MODULE: off
strategy: strategy:
matrix: matrix:
go-version: [1.16.x] go-version: [1.17.x]
os: [ubuntu-latest] os: [ubuntu-latest]
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
steps: steps:

View File

@ -9,7 +9,7 @@ jobs:
publish: publish:
strategy: strategy:
matrix: matrix:
go-version: [ 1.16.x ] go-version: [ 1.17.x ]
os: [ ubuntu-latest ] os: [ ubuntu-latest ]
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
steps: steps:

View File

@ -1,5 +1,6 @@
# faasd - a lightweight & portable faas engine # faasd - a lightweight & portable faas engine
[![Sponsor this](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&link=https://github.com/sponsors/openfaas)](https://github.com/sponsors/openfaas)
[![Build Status](https://github.com/openfaas/faasd/workflows/build/badge.svg?branch=master)](https://github.com/openfaas/faasd/actions) [![Build Status](https://github.com/openfaas/faasd/workflows/build/badge.svg?branch=master)](https://github.com/openfaas/faasd/actions)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![OpenFaaS](https://img.shields.io/badge/openfaas-serverless-blue.svg)](https://www.openfaas.com) [![OpenFaaS](https://img.shields.io/badge/openfaas-serverless-blue.svg)](https://www.openfaas.com)
@ -44,7 +45,7 @@ Additional resources:
Most importantly, it's easy to manage so you can set it up and leave it alone to run your functions. Most importantly, it's easy to manage so you can set it up and leave it alone to run your functions.
![demo](https://pbs.twimg.com/media/EPNQz00W4AEwDxM?format=jpg&name=small) [![demo](https://pbs.twimg.com/media/EPNQz00W4AEwDxM?format=jpg&name=medium)](https://www.youtube.com/watch?v=WX1tZoSXy8E)
> Demo of faasd running asynchronous functions > Demo of faasd running asynchronous functions
@ -166,4 +167,6 @@ For completed features, WIP and upcoming roadmap see:
See [ROADMAP.md](docs/ROADMAP.md) See [ROADMAP.md](docs/ROADMAP.md)
Want to build a patch without setting up a complete development environment? See [docs/PATCHES.md](docs/PATCHES.md)
Are you looking to hack on faasd? Follow the [developer instructions](docs/DEV.md) for a manual installation, or use the `hack/install.sh` script and pick up from there. Are you looking to hack on faasd? Follow the [developer instructions](docs/DEV.md) for a manual installation, or use the `hack/install.sh` script and pick up from there.

View File

@ -18,8 +18,8 @@ runcmd:
- mkdir -p /opt/cni/bin - mkdir -p /opt/cni/bin
- curl -sSL https://github.com/containernetworking/plugins/releases/download/v0.8.5/cni-plugins-linux-amd64-v0.8.5.tgz | tar -xz -C /opt/cni/bin - curl -sSL https://github.com/containernetworking/plugins/releases/download/v0.8.5/cni-plugins-linux-amd64-v0.8.5.tgz | tar -xz -C /opt/cni/bin
- mkdir -p /go/src/github.com/openfaas/ - mkdir -p /go/src/github.com/openfaas/
- cd /go/src/github.com/openfaas/ && git clone --depth 1 --branch 0.12.5 https://github.com/openfaas/faasd - cd /go/src/github.com/openfaas/ && git clone --depth 1 --branch 0.13.0 https://github.com/openfaas/faasd
- curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.12.5/faasd" --output "/usr/local/bin/faasd" && chmod a+x "/usr/local/bin/faasd" - curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.13.0/faasd" --output "/usr/local/bin/faasd" && chmod a+x "/usr/local/bin/faasd"
- cd /go/src/github.com/openfaas/faasd/ && /usr/local/bin/faasd install - cd /go/src/github.com/openfaas/faasd/ && /usr/local/bin/faasd install
- systemctl status -l containerd --no-pager - systemctl status -l containerd --no-pager
- journalctl -u faasd-provider --no-pager - journalctl -u faasd-provider --no-pager

View File

@ -109,7 +109,16 @@ func binExists(folder, name string) error {
} }
return nil return nil
} }
func ensureSecretsDir(folder string) error {
if _, err := os.Stat(folder); err != nil {
err = os.MkdirAll(folder, secretDirPermission)
if err != nil {
return err
}
}
return nil
}
func ensureWorkingDir(folder string) error { func ensureWorkingDir(folder string) error {
if _, err := os.Stat(folder); err != nil { if _, err := os.Stat(folder); err != nil {
err = os.MkdirAll(folder, workingDirectoryPermission) err = os.MkdirAll(folder, workingDirectoryPermission)

View File

@ -1,8 +1,8 @@
package cmd package cmd
import ( import (
"encoding/json"
"fmt" "fmt"
"io"
"io/ioutil" "io/ioutil"
"log" "log"
"net/http" "net/http"
@ -14,6 +14,7 @@ import (
"github.com/openfaas/faas-provider/logs" "github.com/openfaas/faas-provider/logs"
"github.com/openfaas/faas-provider/proxy" "github.com/openfaas/faas-provider/proxy"
"github.com/openfaas/faas-provider/types" "github.com/openfaas/faas-provider/types"
faasd "github.com/openfaas/faasd/pkg"
"github.com/openfaas/faasd/pkg/cninetwork" "github.com/openfaas/faasd/pkg/cninetwork"
faasdlogs "github.com/openfaas/faasd/pkg/logs" faasdlogs "github.com/openfaas/faasd/pkg/logs"
"github.com/openfaas/faasd/pkg/provider/config" "github.com/openfaas/faasd/pkg/provider/config"
@ -21,6 +22,8 @@ import (
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
const secretDirPermission = 0755
func makeProviderCmd() *cobra.Command { func makeProviderCmd() *cobra.Command {
var command = &cobra.Command{ var command = &cobra.Command{
Use: "provider", Use: "provider",
@ -82,20 +85,25 @@ func makeProviderCmd() *cobra.Command {
invokeResolver := handlers.NewInvokeResolver(client) invokeResolver := handlers.NewInvokeResolver(client)
userSecretPath := path.Join(wd, "secrets") baseUserSecretsPath := path.Join(wd, "secrets")
if err := moveSecretsToDefaultNamespaceSecrets(
baseUserSecretsPath,
faasd.DefaultFunctionNamespace); err != nil {
return err
}
bootstrapHandlers := types.FaaSHandlers{ bootstrapHandlers := types.FaaSHandlers{
FunctionProxy: proxy.NewHandlerFunc(*config, invokeResolver), FunctionProxy: proxy.NewHandlerFunc(*config, invokeResolver),
DeleteHandler: handlers.MakeDeleteHandler(client, cni), DeleteHandler: handlers.MakeDeleteHandler(client, cni),
DeployHandler: handlers.MakeDeployHandler(client, cni, userSecretPath, alwaysPull), DeployHandler: handlers.MakeDeployHandler(client, cni, baseUserSecretsPath, alwaysPull),
FunctionReader: handlers.MakeReadHandler(client), FunctionReader: handlers.MakeReadHandler(client),
ReplicaReader: handlers.MakeReplicaReaderHandler(client), ReplicaReader: handlers.MakeReplicaReaderHandler(client),
ReplicaUpdater: handlers.MakeReplicaUpdateHandler(client, cni), ReplicaUpdater: handlers.MakeReplicaUpdateHandler(client, cni),
UpdateHandler: handlers.MakeUpdateHandler(client, cni, userSecretPath, alwaysPull), UpdateHandler: handlers.MakeUpdateHandler(client, cni, baseUserSecretsPath, alwaysPull),
HealthHandler: func(w http.ResponseWriter, r *http.Request) {}, HealthHandler: func(w http.ResponseWriter, r *http.Request) {},
InfoHandler: handlers.MakeInfoHandler(Version, GitCommit), InfoHandler: handlers.MakeInfoHandler(Version, GitCommit),
ListNamespaceHandler: listNamespaces(), ListNamespaceHandler: handlers.MakeNamespacesLister(client),
SecretHandler: handlers.MakeSecretHandler(client, userSecretPath), SecretHandler: handlers.MakeSecretHandler(client, baseUserSecretsPath),
LogHandler: logs.NewLogHandlerFunc(faasdlogs.New(), config.ReadTimeout), LogHandler: logs.NewLogHandlerFunc(faasdlogs.New(), config.ReadTimeout),
} }
@ -107,10 +115,62 @@ func makeProviderCmd() *cobra.Command {
return command return command
} }
func listNamespaces() func(w http.ResponseWriter, r *http.Request) { /*
return func(w http.ResponseWriter, r *http.Request) { * Mutiple namespace support was added after release 0.13.0
list := []string{""} * Function will help users to migrate on multiple namespace support of faasd
out, _ := json.Marshal(list) */
w.Write(out) func moveSecretsToDefaultNamespaceSecrets(baseSecretPath string, defaultNamespace string) error {
newSecretPath := path.Join(baseSecretPath, defaultNamespace)
err := ensureSecretsDir(newSecretPath)
if err != nil {
return err
} }
files, err := ioutil.ReadDir(baseSecretPath)
if err != nil {
return err
}
for _, f := range files {
if !f.IsDir() {
newPath := path.Join(newSecretPath, f.Name())
// A non-nil error means the file wasn't found in the
// destination path
if _, err := os.Stat(newPath); err != nil {
oldPath := path.Join(baseSecretPath, f.Name())
if err := copyFile(oldPath, newPath); err != nil {
return err
}
log.Printf("[Migration] Copied %s to %s", oldPath, newPath)
}
}
}
return nil
}
func copyFile(src, dst string) error {
inputFile, err := os.Open(src)
if err != nil {
return fmt.Errorf("opening %s failed %w", src, err)
}
defer inputFile.Close()
outputFile, err := os.OpenFile(dst, os.O_CREATE|os.O_WRONLY|os.O_APPEND, secretDirPermission)
if err != nil {
return fmt.Errorf("opening %s failed %w", dst, err)
}
defer outputFile.Close()
// Changed from os.Rename due to issue in #201
if _, err := io.Copy(outputFile, inputFile); err != nil {
return fmt.Errorf("writing into %s failed %w", outputFile.Name(), err)
}
return nil
} }

View File

@ -20,21 +20,36 @@ services:
nats: nats:
image: docker.io/library/nats-streaming:0.22.0 image: docker.io/library/nats-streaming:0.22.0
# nobody
user: "65534"
command: command:
- "/nats-streaming-server" - "/nats-streaming-server"
- "-m" - "-m"
- "8222" - "8222"
- "--store=memory" - "--store=file"
- "--dir=/nats"
- "--cluster_id=faas-cluster" - "--cluster_id=faas-cluster"
volumes:
# Data directory
- type: bind
source: ./nats
target: /nats
# ports: # ports:
# - "127.0.0.1:8222:8222" # - "127.0.0.1:8222:8222"
prometheus: prometheus:
image: docker.io/prom/prometheus:v2.14.0 image: docker.io/prom/prometheus:v2.14.0
# nobody
user: "65534"
volumes: volumes:
# Config directory
- type: bind - type: bind
source: ./prometheus.yml source: ./prometheus.yml
target: /etc/prometheus/prometheus.yml target: /etc/prometheus/prometheus.yml
# Data directory
- type: bind
source: ./prometheus
target: /prometheus
cap_add: cap_add:
- CAP_NET_RAW - CAP_NET_RAW
ports: ports:

View File

@ -1,7 +1,11 @@
## Instructions for hacking on faasd itself ## Instructions for building and testing faasd locally
> Note: if you're just wanting to try out faasd, then it's likely that you're on the wrong page. This is a detailed set of instructions for those wanting to contribute or customise faasd. Feel free to go back to the homepage and pick a tutorial instead. > Note: if you're just wanting to try out faasd, then it's likely that you're on the wrong page. This is a detailed set of instructions for those wanting to contribute or customise faasd. Feel free to go back to the homepage and pick a tutorial instead.
Do you want to help the community test a pull request?
See these instructions instead: [Testing patches](/docs/PATCHES.md)
### Pre-reqs ### Pre-reqs
> It's recommended that you do not install Docker on the same host as faasd, since 1) they may both use different versions of containerd and 2) docker's networking rules can disrupt faasd's networking. When using faasd - make your faasd server a faasd server, and build container image on your laptop or in a CI pipeline. > It's recommended that you do not install Docker on the same host as faasd, since 1) they may both use different versions of containerd and 2) docker's networking rules can disrupt faasd's networking. When using faasd - make your faasd server a faasd server, and build container image on your laptop or in a CI pipeline.
@ -233,7 +237,7 @@ export SUFFIX="-armhf"
export SUFFIX="-arm64" export SUFFIX="-arm64"
# Then download # Then download
curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.12.5/faasd$SUFFIX" \ curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.13.0/faasd$SUFFIX" \
-o "/tmp/faasd" \ -o "/tmp/faasd" \
&& chmod +x "/tmp/faasd" && chmod +x "/tmp/faasd"
sudo mv /tmp/faasd /usr/local/bin/ sudo mv /tmp/faasd /usr/local/bin/

88
docs/PATCHES.md Normal file
View File

@ -0,0 +1,88 @@
## Instructions for testing a patch for faasd
### Launch a virtual machine
You can use any kind of Linux virtual machine, Ubuntu 20.04 is recommended.
Launch a cloud VM or use [Multipass](https://multipass.run), which is free to use an can be run locally. A Raspberry Pi 3 or 4 could also be used, but will need you to run `make dist` to cross compile a valid binary.
### Copy over your SSH key
Your SSH key will be used, so that you can copy a new faasd binary over to the host.
```bash
multipass launch \
--mem 4G \
-c 2 \
-n faasd
# Then access its shell
multipass shell faasd
# Edit .ssh/authorized_keys
# Add .ssh/id_rsa.pub from your host and save the file
```
### Install faasd on the VM
You start off with the upstream version of faasd on the host, then add the new version over the top later on.
```bash
cd /tmp/
git clone https://github.com/openfaas/faasd --depth=1
cd faasd/hack
./install.sh
# Run the login command given to you at the end of the script
```
Get the multipass IP address:
```bash
export IP=$(multipass info faasd --format json| jq -r '.info.faasd.ipv4[0]')
```
### Build a new faasd binary with the patch
Check out faasd on your local computer
```bash
git clone https://github.com/openfaas/faasd
cd faasd
gh pr checkout #PR_NUMBER_HERE
GOOS=linux go build
# You can also run "make dist" which is slower, but includes
# a version and binaries for other platforms such as the Raspberry Pi
```
### Copy it over to the VM
Now build a new faasd binary and copy it to the VM:
```bash
scp faasd ubuntu@$IP:~/
```
Now deploy the new version on the VM:
```bash
killall -9 faasd-linux; killall -9 faasd-linux ; mv ./faasd-linux /usr/local/bin/faasd
```
### Check it worked and test that patch
Now run a command with `faas-cli` such as:
* `faas-cli list`
* `faas-cli version`
See the testing instructions on the PR and run through those steps.
Post your results on GitHub to assist the creator of the pull request.
You can see how to get the logs for various components using the [eBook Serverless For Everyone Else](https://gumroad.com/l/serverless-for-everyone-else), or by consulting the [DEV.md](/docs/DEV.md) guide.

View File

@ -75,10 +75,10 @@ sudo systemctl restart faasd
Should have: Should have:
* [ ] Resolve core services from functions by populating/sharing `/etc/hosts` between `faasd` and `faasd-provider` * [ ] Offer a recommendation or implement a strategy for faasd replication/HA
* [ ] Docs or examples on how to use the various connectors and connector-sdk
* [ ] Monitor and restart any of the core components at runtime if the container stops * [ ] Monitor and restart any of the core components at runtime if the container stops
* [ ] Asynchronous deletion instead of synchronous * [ ] Asynchronous function deletion instead of synchronous
* [ ] Asynchronous function start-up instead of synchronous
Nice to Have: Nice to Have:
@ -89,6 +89,8 @@ Nice to Have:
### Completed ### Completed
* [x] Docs or examples on how to use the various event connectors (Yes in the eBook)
* [x] Resolve core services from functions by populating/sharing `/etc/hosts` between `faasd` and `faasd-provider`
* [x] Provide a cloud-init configuration for faasd bootstrap * [x] Provide a cloud-init configuration for faasd bootstrap
* [x] Configure core services from a docker-compose.yaml file * [x] Configure core services from a docker-compose.yaml file
* [x] Store and fetch logs from the journal * [x] Store and fetch logs from the journal

66
docs/bootstrap/.terraform.lock.hcl generated Normal file
View File

@ -0,0 +1,66 @@
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/digitalocean/digitalocean" {
version = "2.11.0"
constraints = "2.11.0"
hashes = [
"h1:/qAnTOSP5KeZkF7wqLai34SKAs7aefulcUA3I8R7rRg=",
"h1:PbXtjUfvxwmkycJ0Y9Dyn66Arrpk5L8/P381SXMx2O0=",
"h1:lXLX9tmuxV7azTHd0xB0FAVrxyfBtotIz5LEJp8YUk0=",
"zh:2191adc79bdfdb3b733e0619e4f391ae91c1631c5dafda42dab561d943651fa4",
"zh:21a4f67e42dcdc10fbd7f8579247594844d09a469a3a54862d565913e4d6121d",
"zh:557d98325fafcf2db91ea6d92f65373a48c4e995a1a7aeb57009661fee675250",
"zh:68c0238cafc37433627e288fcd2c7e14f4f0afdd50b4f265d8d1f1addab6f19f",
"zh:7e6d69720734455eb1c69880f049650276089b7fa09085e130d224abaeec887a",
"zh:95bd93a696ec050c1cb5e724498fd12b1d69760d01e97c869be3252025691434",
"zh:b1b075049e33aa08c032f41a497351c9894f16287a4449032d8b805bc6dcb596",
"zh:ba91aa853372c828f808c09dbab2a5bc9493a7cf93210d1487f9637b2cac8ca4",
"zh:bc43d27dfe014266697c2ac259f4311300391aa6aa7c5d23e382fe296df938d5",
"zh:d3a04d2c76bfc1f46a117b1af7870a97353319ee8f924a37fe77861519f59525",
"zh:d3da997c05a653df6cabb912c6c05ceb6bf77219b699f04daf44fd795c81c6ed",
"zh:edd0659021b6634acf0f581d1be1985a81fcd1182e3ccb43de6eac6c43be9ab4",
"zh:f588ace57b6c35d509ecaa7136e6a8049d227b0674104a1f958359b84862d8e3",
"zh:f894ed195a3b9ebbfa1ba7c5d71be06df3a96d783ff064d22dd693ace34d638e",
"zh:fb6b0d4b111fafdcb3bb9a7dbab88e2110a6ce6324de64ecf62933ee8b651ccf",
]
}
provider "registry.terraform.io/hashicorp/random" {
version = "3.1.0"
hashes = [
"h1:BZMEPucF+pbu9gsPk0G0BHx7YP04+tKdq2MrRDF1EDM=",
"h1:EPIax4Ftp2SNdB9pUfoSjxoueDoLc/Ck3EUoeX0Dvsg=",
"h1:rKYu5ZUbXwrLG1w81k7H3nce/Ys6yAxXhWcbtk36HjY=",
"zh:2bbb3339f0643b5daa07480ef4397bd23a79963cc364cdfbb4e86354cb7725bc",
"zh:3cd456047805bf639fbf2c761b1848880ea703a054f76db51852008b11008626",
"zh:4f251b0eda5bb5e3dc26ea4400dba200018213654b69b4a5f96abee815b4f5ff",
"zh:7011332745ea061e517fe1319bd6c75054a314155cb2c1199a5b01fe1889a7e2",
"zh:738ed82858317ccc246691c8b85995bc125ac3b4143043219bd0437adc56c992",
"zh:7dbe52fac7bb21227acd7529b487511c91f4107db9cc4414f50d04ffc3cab427",
"zh:a3a9251fb15f93e4cfc1789800fc2d7414bbc18944ad4c5c98f466e6477c42bc",
"zh:a543ec1a3a8c20635cf374110bd2f87c07374cf2c50617eee2c669b3ceeeaa9f",
"zh:d9ab41d556a48bd7059f0810cf020500635bfc696c9fc3adab5ea8915c1d886b",
"zh:d9e13427a7d011dbd654e591b0337e6074eef8c3b9bb11b2e39eaaf257044fd7",
"zh:f7605bd1437752114baf601bdf6931debe6dc6bfe3006eb7e9bb9080931dca8a",
]
}
provider "registry.terraform.io/hashicorp/template" {
version = "2.2.0"
hashes = [
"h1:0wlehNaxBX7GJQnPfQwTNvvAf38Jm0Nv7ssKGMaG6Og=",
"h1:94qn780bi1qjrbC3uQtjJh3Wkfwd5+tTtJHOb7KTg9w=",
"h1:LN84cu+BZpVRvYlCzrbPfCRDaIelSyEx/W9Iwwgbnn4=",
"zh:01702196f0a0492ec07917db7aaa595843d8f171dc195f4c988d2ffca2a06386",
"zh:09aae3da826ba3d7df69efeb25d146a1de0d03e951d35019a0f80e4f58c89b53",
"zh:09ba83c0625b6fe0a954da6fbd0c355ac0b7f07f86c91a2a97849140fea49603",
"zh:0e3a6c8e16f17f19010accd0844187d524580d9fdb0731f675ffcf4afba03d16",
"zh:45f2c594b6f2f34ea663704cc72048b212fe7d16fb4cfd959365fa997228a776",
"zh:77ea3e5a0446784d77114b5e851c970a3dde1e08fa6de38210b8385d7605d451",
"zh:8a154388f3708e3df5a69122a23bdfaf760a523788a5081976b3d5616f7d30ae",
"zh:992843002f2db5a11e626b3fc23dc0c87ad3729b3b3cff08e32ffb3df97edbde",
"zh:ad906f4cebd3ec5e43d5cd6dc8f4c5c9cc3b33d2243c89c5fc18f97f7277b51d",
"zh:c979425ddb256511137ecd093e23283234da0154b7fa8b21c2687182d9aea8b2",
]
}

View File

@ -5,11 +5,18 @@
3) Clone this gist using the URL from the address bar 3) Clone this gist using the URL from the address bar
4) Run `terraform init` 4) Run `terraform init`
5) Run `terraform apply -var="do_token=$(cat $HOME/digitalocean-access-token)"` 5) Run `terraform apply -var="do_token=$(cat $HOME/digitalocean-access-token)"`
6) View the output for the login command and gateway URL i.e. 6) View the output for the gateway URL
``` ```
gateway_url = http://178.128.39.201:8080/ gateway_url = http://178.128.39.201:8080/
```
7) View the output for sensitive data via `terraform output` command
```bash
terraform output login_cmd
login_cmd = faas-cli login -g http://178.128.39.201:8080/ -p rvIU49CEcFcHmqxj login_cmd = faas-cli login -g http://178.128.39.201:8080/ -p rvIU49CEcFcHmqxj
terraform output password
password = rvIU49CEcFcHmqxj password = rvIU49CEcFcHmqxj
``` ```

View File

@ -1,8 +1,4 @@
#cloud-config #cloud-config
ssh_authorized_keys:
## Note: Replace with your own public key
- ${ssh_key}
package_update: true package_update: true
packages: packages:
@ -20,8 +16,8 @@ runcmd:
- mkdir -p /var/lib/faasd/secrets/ - mkdir -p /var/lib/faasd/secrets/
- echo ${gw_password} > /var/lib/faasd/secrets/basic-auth-password - echo ${gw_password} > /var/lib/faasd/secrets/basic-auth-password
- echo admin > /var/lib/faasd/secrets/basic-auth-user - echo admin > /var/lib/faasd/secrets/basic-auth-user
- cd /go/src/github.com/openfaas/ && git clone --depth 1 --branch 0.12.5 https://github.com/openfaas/faasd - cd /go/src/github.com/openfaas/ && git clone --depth 1 --branch 0.13.0 https://github.com/openfaas/faasd
- curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.12.5/faasd" --output "/usr/local/bin/faasd" && chmod a+x "/usr/local/bin/faasd" - curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.13.0/faasd" --output "/usr/local/bin/faasd" && chmod a+x "/usr/local/bin/faasd"
- cd /go/src/github.com/openfaas/faasd/ && /usr/local/bin/faasd install - cd /go/src/github.com/openfaas/faasd/ && /usr/local/bin/faasd install
- systemctl status -l containerd --no-pager - systemctl status -l containerd --no-pager
- journalctl -u faasd-provider --no-pager - journalctl -u faasd-provider --no-pager

View File

@ -0,0 +1,66 @@
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/digitalocean/digitalocean" {
version = "2.11.0"
constraints = "2.11.0"
hashes = [
"h1:/qAnTOSP5KeZkF7wqLai34SKAs7aefulcUA3I8R7rRg=",
"h1:PbXtjUfvxwmkycJ0Y9Dyn66Arrpk5L8/P381SXMx2O0=",
"h1:lXLX9tmuxV7azTHd0xB0FAVrxyfBtotIz5LEJp8YUk0=",
"zh:2191adc79bdfdb3b733e0619e4f391ae91c1631c5dafda42dab561d943651fa4",
"zh:21a4f67e42dcdc10fbd7f8579247594844d09a469a3a54862d565913e4d6121d",
"zh:557d98325fafcf2db91ea6d92f65373a48c4e995a1a7aeb57009661fee675250",
"zh:68c0238cafc37433627e288fcd2c7e14f4f0afdd50b4f265d8d1f1addab6f19f",
"zh:7e6d69720734455eb1c69880f049650276089b7fa09085e130d224abaeec887a",
"zh:95bd93a696ec050c1cb5e724498fd12b1d69760d01e97c869be3252025691434",
"zh:b1b075049e33aa08c032f41a497351c9894f16287a4449032d8b805bc6dcb596",
"zh:ba91aa853372c828f808c09dbab2a5bc9493a7cf93210d1487f9637b2cac8ca4",
"zh:bc43d27dfe014266697c2ac259f4311300391aa6aa7c5d23e382fe296df938d5",
"zh:d3a04d2c76bfc1f46a117b1af7870a97353319ee8f924a37fe77861519f59525",
"zh:d3da997c05a653df6cabb912c6c05ceb6bf77219b699f04daf44fd795c81c6ed",
"zh:edd0659021b6634acf0f581d1be1985a81fcd1182e3ccb43de6eac6c43be9ab4",
"zh:f588ace57b6c35d509ecaa7136e6a8049d227b0674104a1f958359b84862d8e3",
"zh:f894ed195a3b9ebbfa1ba7c5d71be06df3a96d783ff064d22dd693ace34d638e",
"zh:fb6b0d4b111fafdcb3bb9a7dbab88e2110a6ce6324de64ecf62933ee8b651ccf",
]
}
provider "registry.terraform.io/hashicorp/random" {
version = "3.1.0"
hashes = [
"h1:BZMEPucF+pbu9gsPk0G0BHx7YP04+tKdq2MrRDF1EDM=",
"h1:EPIax4Ftp2SNdB9pUfoSjxoueDoLc/Ck3EUoeX0Dvsg=",
"h1:rKYu5ZUbXwrLG1w81k7H3nce/Ys6yAxXhWcbtk36HjY=",
"zh:2bbb3339f0643b5daa07480ef4397bd23a79963cc364cdfbb4e86354cb7725bc",
"zh:3cd456047805bf639fbf2c761b1848880ea703a054f76db51852008b11008626",
"zh:4f251b0eda5bb5e3dc26ea4400dba200018213654b69b4a5f96abee815b4f5ff",
"zh:7011332745ea061e517fe1319bd6c75054a314155cb2c1199a5b01fe1889a7e2",
"zh:738ed82858317ccc246691c8b85995bc125ac3b4143043219bd0437adc56c992",
"zh:7dbe52fac7bb21227acd7529b487511c91f4107db9cc4414f50d04ffc3cab427",
"zh:a3a9251fb15f93e4cfc1789800fc2d7414bbc18944ad4c5c98f466e6477c42bc",
"zh:a543ec1a3a8c20635cf374110bd2f87c07374cf2c50617eee2c669b3ceeeaa9f",
"zh:d9ab41d556a48bd7059f0810cf020500635bfc696c9fc3adab5ea8915c1d886b",
"zh:d9e13427a7d011dbd654e591b0337e6074eef8c3b9bb11b2e39eaaf257044fd7",
"zh:f7605bd1437752114baf601bdf6931debe6dc6bfe3006eb7e9bb9080931dca8a",
]
}
provider "registry.terraform.io/hashicorp/template" {
version = "2.2.0"
hashes = [
"h1:0wlehNaxBX7GJQnPfQwTNvvAf38Jm0Nv7ssKGMaG6Og=",
"h1:94qn780bi1qjrbC3uQtjJh3Wkfwd5+tTtJHOb7KTg9w=",
"h1:LN84cu+BZpVRvYlCzrbPfCRDaIelSyEx/W9Iwwgbnn4=",
"zh:01702196f0a0492ec07917db7aaa595843d8f171dc195f4c988d2ffca2a06386",
"zh:09aae3da826ba3d7df69efeb25d146a1de0d03e951d35019a0f80e4f58c89b53",
"zh:09ba83c0625b6fe0a954da6fbd0c355ac0b7f07f86c91a2a97849140fea49603",
"zh:0e3a6c8e16f17f19010accd0844187d524580d9fdb0731f675ffcf4afba03d16",
"zh:45f2c594b6f2f34ea663704cc72048b212fe7d16fb4cfd959365fa997228a776",
"zh:77ea3e5a0446784d77114b5e851c970a3dde1e08fa6de38210b8385d7605d451",
"zh:8a154388f3708e3df5a69122a23bdfaf760a523788a5081976b3d5616f7d30ae",
"zh:992843002f2db5a11e626b3fc23dc0c87ad3729b3b3cff08e32ffb3df97edbde",
"zh:ad906f4cebd3ec5e43d5cd6dc8f4c5c9cc3b33d2243c89c5fc18f97f7277b51d",
"zh:c979425ddb256511137ecd093e23283234da0154b7fa8b21c2687182d9aea8b2",
]
}

View File

@ -27,10 +27,20 @@
``` ```
droplet_ip = 178.128.39.201 droplet_ip = 178.128.39.201
gateway_url = https://faasd.example.com/ gateway_url = https://faasd.example.com/
login_cmd = faas-cli login -g https://faasd.example.com/ -p rvIU49CEcFcHmqxj ```
8) View the output for sensitive data via `terraform output` command
```bash
terraform output login_cmd
login_cmd = faas-cli login -g http://178.128.39.201:8080/ -p rvIU49CEcFcHmqxj
terraform output password
password = rvIU49CEcFcHmqxj password = rvIU49CEcFcHmqxj
``` ```
8) Use your browser to access the OpenFaaS interface
9) Use your browser to access the OpenFaaS interface
Note that the user-data may take a couple of minutes to come up since it will be pulling in various components and preparing the machine. Note that the user-data may take a couple of minutes to come up since it will be pulling in various components and preparing the machine.
Also take into consideration the DNS propagation time for the new DNS record. Also take into consideration the DNS propagation time for the new DNS record.

View File

@ -1,7 +1,4 @@
#cloud-config #cloud-config
ssh_authorized_keys:
- ${ssh_key}
groups: groups:
- caddy - caddy
@ -41,8 +38,8 @@ runcmd:
- mkdir -p /var/lib/faasd/secrets/ - mkdir -p /var/lib/faasd/secrets/
- echo ${gw_password} > /var/lib/faasd/secrets/basic-auth-password - echo ${gw_password} > /var/lib/faasd/secrets/basic-auth-password
- echo admin > /var/lib/faasd/secrets/basic-auth-user - echo admin > /var/lib/faasd/secrets/basic-auth-user
- cd /go/src/github.com/openfaas/ && git clone --depth 1 --branch 0.12.5 https://github.com/openfaas/faasd - cd /go/src/github.com/openfaas/ && git clone --depth 1 --branch 0.13.0 https://github.com/openfaas/faasd
- curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.12.5/faasd" --output "/usr/local/bin/faasd" && chmod a+x "/usr/local/bin/faasd" - curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.13.0/faasd" --output "/usr/local/bin/faasd" && chmod a+x "/usr/local/bin/faasd"
- cd /go/src/github.com/openfaas/faasd/ && /usr/local/bin/faasd install - cd /go/src/github.com/openfaas/faasd/ && /usr/local/bin/faasd install
- systemctl status -l containerd --no-pager - systemctl status -l containerd --no-pager
- journalctl -u faasd-provider --no-pager - journalctl -u faasd-provider --no-pager

View File

@ -1,5 +1,11 @@
terraform { terraform {
required_version = ">= 0.12" required_version = ">= 1.0.4"
required_providers {
digitalocean = {
source = "digitalocean/digitalocean"
version = "2.11.0"
}
}
} }
variable "do_token" { variable "do_token" {
@ -10,7 +16,7 @@ variable "do_domain" {
} }
variable "do_subdomain" { variable "do_subdomain" {
description = "Your public subdomain" description = "Your public subdomain"
default = "faasd" default = "faasd"
} }
variable "letsencrypt_email" { variable "letsencrypt_email" {
description = "Email used to order a certificate from Letsencrypt" description = "Email used to order a certificate from Letsencrypt"
@ -32,41 +38,44 @@ provider "digitalocean" {
token = var.do_token token = var.do_token
} }
data "local_file" "ssh_key"{
filename = pathexpand(var.ssh_key_file)
}
resource "random_password" "password" { resource "random_password" "password" {
length = 16 length = 16
special = true special = true
override_special = "_-#" override_special = "_-#"
} }
data "template_file" "cloud_init" { data "template_file" "cloud_init" {
template = "${file("cloud-config.tpl")}" template = file("cloud-config.tpl")
vars = { vars = {
gw_password=random_password.password.result, gw_password = random_password.password.result,
ssh_key=data.local_file.ssh_key.content, faasd_domain_name = "${var.do_subdomain}.${var.do_domain}"
faasd_domain_name="${var.do_subdomain}.${var.do_domain}" letsencrypt_email = var.letsencrypt_email
letsencrypt_email=var.letsencrypt_email }
} }
resource "digitalocean_ssh_key" "faasd_ssh_key" {
name = "ssh-key"
public_key = file(var.ssh_key_file)
} }
resource "digitalocean_droplet" "faasd" { resource "digitalocean_droplet" "faasd" {
region = var.do_region region = var.do_region
image = "ubuntu-18-04-x64" image = "ubuntu-18-04-x64"
name = "faasd" name = "faasd"
size = "s-1vcpu-1gb" size = "s-1vcpu-1gb"
user_data = data.template_file.cloud_init.rendered user_data = data.template_file.cloud_init.rendered
ssh_keys = [
digitalocean_ssh_key.faasd_ssh_key.id
]
} }
resource "digitalocean_record" "faasd" { resource "digitalocean_record" "faasd" {
domain = var.do_domain domain = var.do_domain
type = "A" type = "A"
name = "faasd" name = var.do_subdomain
value = digitalocean_droplet.faasd.ipv4_address value = digitalocean_droplet.faasd.ipv4_address
# Only creates record if do_create_record is true # Only creates record if do_create_record is true
count = var.do_create_record == true ? 1 : 0 count = var.do_create_record == true ? 1 : 0
} }
output "droplet_ip" { output "droplet_ip" {
@ -78,9 +87,11 @@ output "gateway_url" {
} }
output "password" { output "password" {
value = random_password.password.result value = random_password.password.result
sensitive = true
} }
output "login_cmd" { output "login_cmd" {
value = "faas-cli login -g https://${var.do_subdomain}.${var.do_domain}/ -p ${random_password.password.result}" value = "faas-cli login -g https://${var.do_subdomain}.${var.do_domain}/ -p ${random_password.password.result}"
sensitive = true
} }

View File

@ -1,5 +1,11 @@
terraform { terraform {
required_version = ">= 0.12" required_version = ">= 1.0.4"
required_providers {
digitalocean = {
source = "digitalocean/digitalocean"
version = "2.11.0"
}
}
} }
variable "do_token" {} variable "do_token" {}
@ -10,25 +16,25 @@ variable "ssh_key_file" {
} }
provider "digitalocean" { provider "digitalocean" {
token = var.do_token token = var.do_token
} }
resource "random_password" "password" { resource "random_password" "password" {
length = 16 length = 16
special = true special = true
override_special = "_-#" override_special = "_-#"
} }
data "local_file" "ssh_key"{ data "template_file" "cloud_init" {
filename = pathexpand(var.ssh_key_file) template = file("cloud-config.tpl")
vars = {
gw_password = random_password.password.result
}
} }
data "template_file" "cloud_init" { resource "digitalocean_ssh_key" "faasd_ssh_key" {
template = "${file("cloud-config.tpl")}" name = "ssh-key"
vars = { public_key = file(var.ssh_key_file)
gw_password=random_password.password.result,
ssh_key=data.local_file.ssh_key.content,
}
} }
resource "digitalocean_droplet" "faasd" { resource "digitalocean_droplet" "faasd" {
@ -38,12 +44,16 @@ resource "digitalocean_droplet" "faasd" {
name = "faasd" name = "faasd"
# Plans: https://developers.digitalocean.com/documentation/changelog/api-v2/new-size-slugs-for-droplet-plan-changes/ # Plans: https://developers.digitalocean.com/documentation/changelog/api-v2/new-size-slugs-for-droplet-plan-changes/
#size = "512mb" #size = "512mb"
size = "s-1vcpu-1gb" size = "s-1vcpu-1gb"
user_data = data.template_file.cloud_init.rendered user_data = data.template_file.cloud_init.rendered
ssh_keys = [
digitalocean_ssh_key.faasd_ssh_key.id
]
} }
output "password" { output "password" {
value = random_password.password.result value = random_password.password.result
sensitive = true
} }
output "gateway_url" { output "gateway_url" {
@ -51,6 +61,7 @@ output "gateway_url" {
} }
output "login_cmd" { output "login_cmd" {
value = "faas-cli login -g http://${digitalocean_droplet.faasd.ipv4_address}:8080/ -p ${random_password.password.result}" value = "faas-cli login -g http://${digitalocean_droplet.faasd.ipv4_address}:8080/ -p ${random_password.password.result}"
sensitive = true
} }

View File

@ -0,0 +1,37 @@
#!/bin/bash
# See pre-reqs:
# https://github.com/alexellis/containerd-arm
export ARCH="arm64"
if [ ! -d "/usr/local/go/bin" ]; then
echo "Downloading Go.."
curl -SLsf https://golang.org/dl/go1.16.6.linux-$ARCH.tar.gz --output /tmp/go.tgz
sudo rm -rf /usr/local/go/
sudo mkdir -p /usr/local/go/
sudo tar -xvf /tmp/go.tgz -C /usr/local/go/ --strip-components=1
else
echo "Go already present, skipping."
fi
export GOPATH=$HOME/go/
export PATH=$PATH:/usr/local/go/bin/
go version
echo "Building containerd"
mkdir -p $GOPATH/src/github.com/containerd
cd $GOPATH/src/github.com/containerd
git clone https://github.com/containerd/containerd
cd containerd
git fetch origin --tags
git checkout v1.5.4
make
sudo make install
sudo containerd --version

View File

@ -1,12 +1,20 @@
#!/bin/bash #!/bin/bash
export ARCH="armv6l" # See pre-reqs:
echo "Downloading Go" # https://github.com/alexellis/containerd-arm
curl -SLsf https://golang.org/dl/go1.16.6.linux-$ARCH.tar.gz --output /tmp/go.tgz export ARCH="arm64"
sudo rm -rf /usr/local/go/
sudo mkdir -p /usr/local/go/ if [ ! -d "/usr/local/go/bin" ]; then
sudo tar -xvf /tmp/go.tgz -C /usr/local/go/ --strip-components=1 echo "Downloading Go.."
curl -SLsf https://golang.org/dl/go1.16.6.linux-$ARCH.tar.gz --output /tmp/go.tgz
sudo rm -rf /usr/local/go/
sudo mkdir -p /usr/local/go/
sudo tar -xvf /tmp/go.tgz -C /usr/local/go/ --strip-components=1
else
echo "Go already present, skipping."
fi
export GOPATH=$HOME/go/ export GOPATH=$HOME/go/
export PATH=$PATH:/usr/local/go/bin/ export PATH=$PATH:/usr/local/go/bin/

View File

@ -1,12 +1,21 @@
#!/bin/bash #!/bin/bash
export ARCH="amd64"
echo "Downloading Go"
curl -SLsf https://golang.org/dl/go1.16.6.linux-$ARCH.tar.gz --output /tmp/go.tgz # See pre-reqs:
sudo rm -rf /usr/local/go/ # https://github.com/alexellis/containerd-arm
sudo mkdir -p /usr/local/go/
sudo tar -xvf /tmp/go.tgz -C /usr/local/go/ --strip-components=1 export ARCH="arm64"
if [ ! -d "/usr/local/go/bin" ]; then
echo "Downloading Go.."
curl -SLsf https://golang.org/dl/go1.16.6.linux-$ARCH.tar.gz --output /tmp/go.tgz
sudo rm -rf /usr/local/go/
sudo mkdir -p /usr/local/go/
sudo tar -xvf /tmp/go.tgz -C /usr/local/go/ --strip-components=1
else
echo "Go already present, skipping."
fi
export GOPATH=$HOME/go/ export GOPATH=$HOME/go/
export PATH=$PATH:/usr/local/go/bin/ export PATH=$PATH:/usr/local/go/bin/

View File

@ -5,6 +5,7 @@ Description=faasd-provider
MemoryLimit=500M MemoryLimit=500M
Environment="secret_mount_path={{.SecretMountPath}}" Environment="secret_mount_path={{.SecretMountPath}}"
Environment="basic_auth=true" Environment="basic_auth=true"
Environment="hosts_dir=/var/lib/faasd"
ExecStart=/usr/local/bin/faasd provider ExecStart=/usr/local/bin/faasd provider
Restart=on-failure Restart=on-failure
RestartSec=10s RestartSec=10s

View File

@ -39,6 +39,10 @@ has_apt_get() {
[ -n "$(command -v apt-get)" ] [ -n "$(command -v apt-get)" ]
} }
has_pacman() {
[ -n "$(command -v pacman)" ]
}
install_required_packages() { install_required_packages() {
if $(has_apt_get); then if $(has_apt_get); then
$SUDO apt-get update -y $SUDO apt-get update -y
@ -46,8 +50,11 @@ install_required_packages() {
elif $(has_yum); then elif $(has_yum); then
$SUDO yum check-update -y $SUDO yum check-update -y
$SUDO yum install -y curl runc $SUDO yum install -y curl runc
elif $(has_pacman); then
$SUDO pacman -Syy
$SUDO pacman -Sy curl runc bridge-utils
else else
fatal "Could not find apt-get or yum. Cannot install dependencies on this OS." fatal "Could not find apt-get, yum, or pacman. Cannot install dependencies on this OS."
exit 1 exit 1
fi fi
} }

View File

@ -1,11 +1,14 @@
package pkg package pkg
const ( const (
// FunctionNamespace is the default containerd namespace functions are created // DefaultFunctionNamespace is the default containerd namespace functions are created
FunctionNamespace = "openfaas-fn" DefaultFunctionNamespace = "openfaas-fn"
// faasdNamespace is the containerd namespace services are created // NamespaceLabel indicates that a namespace is managed by faasd
faasdNamespace = "openfaas" NamespaceLabel = "openfaas"
// FaasdNamespace is the containerd namespace services are created
FaasdNamespace = "openfaas"
faasServicesPullAlways = false faasServicesPullAlways = false

View File

@ -71,7 +71,7 @@ func buildCmd(ctx context.Context, req logs.Request) *exec.Cmd {
namespace := req.Namespace namespace := req.Namespace
if namespace == "" { if namespace == "" {
namespace = faasd.FunctionNamespace namespace = faasd.DefaultFunctionNamespace
} }
// find the description of the fields here // find the description of the fields here

View File

@ -13,7 +13,6 @@ import (
gocni "github.com/containerd/go-cni" gocni "github.com/containerd/go-cni"
"github.com/openfaas/faas/gateway/requests" "github.com/openfaas/faas/gateway/requests"
faasd "github.com/openfaas/faasd/pkg"
cninetwork "github.com/openfaas/faasd/pkg/cninetwork" cninetwork "github.com/openfaas/faasd/pkg/cninetwork"
"github.com/openfaas/faasd/pkg/service" "github.com/openfaas/faasd/pkg/service"
) )
@ -41,9 +40,23 @@ func MakeDeleteHandler(client *containerd.Client, cni gocni.CNI) func(w http.Res
return return
} }
lookupNamespace := getRequestNamespace(readNamespaceFromQuery(r))
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client, lookupNamespace)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
name := req.FunctionName name := req.FunctionName
function, err := GetFunction(client, name) function, err := GetFunction(client, name, lookupNamespace)
if err != nil { if err != nil {
msg := fmt.Sprintf("service %s not found", name) msg := fmt.Sprintf("service %s not found", name)
log.Printf("[Delete] %s\n", msg) log.Printf("[Delete] %s\n", msg)
@ -51,7 +64,7 @@ func MakeDeleteHandler(client *containerd.Client, cni gocni.CNI) func(w http.Res
return return
} }
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace) ctx := namespaces.WithNamespace(context.Background(), lookupNamespace)
// TODO: this needs to still happen if the task is paused // TODO: this needs to still happen if the task is paused
if function.replicas != 0 { if function.replicas != 0 {

View File

@ -20,7 +20,6 @@ import (
"github.com/docker/distribution/reference" "github.com/docker/distribution/reference"
"github.com/opencontainers/runtime-spec/specs-go" "github.com/opencontainers/runtime-spec/specs-go"
"github.com/openfaas/faas-provider/types" "github.com/openfaas/faas-provider/types"
faasd "github.com/openfaas/faasd/pkg"
cninetwork "github.com/openfaas/faasd/pkg/cninetwork" cninetwork "github.com/openfaas/faasd/pkg/cninetwork"
"github.com/openfaas/faasd/pkg/service" "github.com/openfaas/faasd/pkg/service"
"github.com/pkg/errors" "github.com/pkg/errors"
@ -52,15 +51,32 @@ func MakeDeployHandler(client *containerd.Client, cni gocni.CNI, secretMountPath
return return
} }
err = validateSecrets(secretMountPath, req.Secrets) namespace := getRequestNamespace(req.Namespace)
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client, namespace)
if err != nil { if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest) http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
namespaceSecretMountPath := getNamespaceSecretMountPath(secretMountPath, namespace)
err = validateSecrets(namespaceSecretMountPath, req.Secrets)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
} }
name := req.Service name := req.Service
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace) ctx := namespaces.WithNamespace(context.Background(), namespace)
deployErr := deploy(ctx, req, client, cni, secretMountPath, alwaysPull) deployErr := deploy(ctx, req, client, cni, namespaceSecretMountPath, alwaysPull)
if deployErr != nil { if deployErr != nil {
log.Printf("[Deploy] error deploying %s, error: %s\n", name, deployErr) log.Printf("[Deploy] error deploying %s, error: %s\n", name, deployErr)
http.Error(w, deployErr.Error(), http.StatusBadRequest) http.Error(w, deployErr.Error(), http.StatusBadRequest)
@ -110,7 +126,7 @@ func deploy(ctx context.Context, req types.FunctionDeployment, client *container
} }
envs := prepareEnv(req.EnvProcess, req.EnvVars) envs := prepareEnv(req.EnvProcess, req.EnvVars)
mounts := getMounts() mounts := getOSMounts()
for _, secret := range req.Secrets { for _, secret := range req.Secrets {
mounts = append(mounts, specs.Mount{ mounts = append(mounts, specs.Mount{
@ -125,7 +141,7 @@ func deploy(ctx context.Context, req types.FunctionDeployment, client *container
labels, err := buildLabels(&req) labels, err := buildLabels(&req)
if err != nil { if err != nil {
return fmt.Errorf("Unable to apply labels to conatiner: %s, error: %s", name, err) return fmt.Errorf("unable to apply labels to container: %s, error: %w", name, err)
} }
var memory *specs.LinuxMemory var memory *specs.LinuxMemory
@ -156,7 +172,7 @@ func deploy(ctx context.Context, req types.FunctionDeployment, client *container
) )
if err != nil { if err != nil {
return fmt.Errorf("unable to create container: %s, error: %s", name, err) return fmt.Errorf("unable to create container: %s, error: %w", name, err)
} }
return createTask(ctx, client, container, cni) return createTask(ctx, client, container, cni)
@ -194,7 +210,7 @@ func createTask(ctx context.Context, client *containerd.Client, container contai
task, taskErr := container.NewTask(ctx, cio.BinaryIO("/usr/local/bin/faasd", nil)) task, taskErr := container.NewTask(ctx, cio.BinaryIO("/usr/local/bin/faasd", nil))
if taskErr != nil { if taskErr != nil {
return fmt.Errorf("unable to start task: %s, error: %s", name, taskErr) return fmt.Errorf("unable to start task: %s, error: %w", name, taskErr)
} }
log.Printf("Container ID: %s\tTask ID %s:\tTask PID: %d\t\n", name, task.ID(), task.Pid()) log.Printf("Container ID: %s\tTask ID %s:\tTask PID: %d\t\n", name, task.ID(), task.Pid())
@ -246,20 +262,28 @@ func prepareEnv(envProcess string, reqEnvVars map[string]string) []string {
return envs return envs
} }
func getMounts() []specs.Mount { // getOSMounts provides a mount for os-specific files such
wd, _ := os.Getwd() // as the hosts file and resolv.conf
func getOSMounts() []specs.Mount {
// Prior to hosts_dir env-var, this value was set to
// os.Getwd()
hostsDir := "/var/lib/faasd"
if v, ok := os.LookupEnv("hosts_dir"); ok && len(v) > 0 {
hostsDir = v
}
mounts := []specs.Mount{} mounts := []specs.Mount{}
mounts = append(mounts, specs.Mount{ mounts = append(mounts, specs.Mount{
Destination: "/etc/resolv.conf", Destination: "/etc/resolv.conf",
Type: "bind", Type: "bind",
Source: path.Join(wd, "resolv.conf"), Source: path.Join(hostsDir, "resolv.conf"),
Options: []string{"rbind", "ro"}, Options: []string{"rbind", "ro"},
}) })
mounts = append(mounts, specs.Mount{ mounts = append(mounts, specs.Mount{
Destination: "/etc/hosts", Destination: "/etc/hosts",
Type: "bind", Type: "bind",
Source: path.Join(wd, "hosts"), Source: path.Join(hostsDir, "hosts"),
Options: []string{"rbind", "ro"}, Options: []string{"rbind", "ro"},
}) })
return mounts return mounts

View File

@ -53,7 +53,7 @@ func Test_BuildLabels_WithAnnotations(t *testing.T) {
} }
if !reflect.DeepEqual(val, tc.result) { if !reflect.DeepEqual(val, tc.result) {
t.Errorf("Got: %s, expected %s", val, tc.result) t.Errorf("Want: %s, got: %s", val, tc.result)
} }
}) })
} }

View File

@ -2,6 +2,7 @@ package handlers
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
"log" "log"
"strings" "strings"
@ -11,9 +12,9 @@ import (
"github.com/containerd/containerd" "github.com/containerd/containerd"
"github.com/containerd/containerd/namespaces" "github.com/containerd/containerd/namespaces"
"github.com/openfaas/faasd/pkg/cninetwork" "github.com/openfaas/faasd/pkg"
faasd "github.com/openfaas/faasd/pkg" faasd "github.com/openfaas/faasd/pkg"
"github.com/openfaas/faasd/pkg/cninetwork"
) )
type Function struct { type Function struct {
@ -28,12 +29,24 @@ type Function struct {
secrets []string secrets []string
envVars map[string]string envVars map[string]string
envProcess string envProcess string
memoryLimit int64
createdAt time.Time createdAt time.Time
} }
// ListFunctions returns a map of all functions with running tasks on namespace // ListFunctions returns a map of all functions with running tasks on namespace
func ListFunctions(client *containerd.Client) (map[string]*Function, error) { func ListFunctions(client *containerd.Client, namespace string) (map[string]*Function, error) {
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace)
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client, namespace)
if err != nil {
return nil, err
}
if !valid {
return nil, errors.New("namespace not valid")
}
ctx := namespaces.WithNamespace(context.Background(), namespace)
functions := make(map[string]*Function) functions := make(map[string]*Function)
containers, err := client.Containers(ctx) containers, err := client.Containers(ctx)
@ -43,7 +56,7 @@ func ListFunctions(client *containerd.Client) (map[string]*Function, error) {
for _, c := range containers { for _, c := range containers {
name := c.ID() name := c.ID()
f, err := GetFunction(client, name) f, err := GetFunction(client, name, namespace)
if err != nil { if err != nil {
log.Printf("error getting function %s: ", name) log.Printf("error getting function %s: ", name)
return functions, err return functions, err
@ -55,13 +68,13 @@ func ListFunctions(client *containerd.Client) (map[string]*Function, error) {
} }
// GetFunction returns a function that matches name // GetFunction returns a function that matches name
func GetFunction(client *containerd.Client, name string) (Function, error) { func GetFunction(client *containerd.Client, name string, namespace string) (Function, error) {
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace) ctx := namespaces.WithNamespace(context.Background(), namespace)
fn := Function{} fn := Function{}
c, err := client.LoadContainer(ctx, name) c, err := client.LoadContainer(ctx, name)
if err != nil { if err != nil {
return Function{}, fmt.Errorf("unable to find function: %s, error %s", name, err) return Function{}, fmt.Errorf("unable to find function: %s, error %w", name, err)
} }
image, err := c.Image(ctx) image, err := c.Image(ctx)
@ -73,26 +86,26 @@ func GetFunction(client *containerd.Client, name string) (Function, error) {
allLabels, labelErr := c.Labels(ctx) allLabels, labelErr := c.Labels(ctx)
if labelErr != nil { if labelErr != nil {
log.Printf("cannot list container %s labels: %s", containerName, labelErr.Error()) log.Printf("cannot list container %s labels: %s", containerName, labelErr)
} }
labels, annotations := buildLabelsAndAnnotations(allLabels) labels, annotations := buildLabelsAndAnnotations(allLabels)
spec, err := c.Spec(ctx) spec, err := c.Spec(ctx)
if err != nil { if err != nil {
return Function{}, fmt.Errorf("unable to load function spec for reading secrets: %s, error %s", name, err) return Function{}, fmt.Errorf("unable to load function %s error: %w", name, err)
} }
info, err := c.Info(ctx) info, err := c.Info(ctx)
if err != nil { if err != nil {
return Function{}, fmt.Errorf("can't load info for: %s, error %s", name, err) return Function{}, fmt.Errorf("can't load info for: %s, error %w", name, err)
} }
envVars, envProcess := readEnvFromProcessEnv(spec.Process.Env) envVars, envProcess := readEnvFromProcessEnv(spec.Process.Env)
secrets := readSecretsFromMounts(spec.Mounts) secrets := readSecretsFromMounts(spec.Mounts)
fn.name = containerName fn.name = containerName
fn.namespace = faasd.FunctionNamespace fn.namespace = namespace
fn.image = image.Name() fn.image = image.Name()
fn.labels = labels fn.labels = labels
fn.annotations = annotations fn.annotations = annotations
@ -100,6 +113,7 @@ func GetFunction(client *containerd.Client, name string) (Function, error) {
fn.envVars = envVars fn.envVars = envVars
fn.envProcess = envProcess fn.envProcess = envProcess
fn.createdAt = info.CreatedAt fn.createdAt = info.CreatedAt
fn.memoryLimit = readMemoryLimitFromSpec(spec)
replicas := 0 replicas := 0
task, err := c.Task(ctx, nil) task, err := c.Task(ctx, nil)
@ -107,7 +121,7 @@ func GetFunction(client *containerd.Client, name string) (Function, error) {
// Task for container exists // Task for container exists
svc, err := task.Status(ctx) svc, err := task.Status(ctx)
if err != nil { if err != nil {
return Function{}, fmt.Errorf("unable to get task status for container: %s %s", name, err) return Function{}, fmt.Errorf("unable to get task status for container: %s %w", name, err)
} }
if svc.Status == "running" { if svc.Status == "running" {
@ -181,3 +195,48 @@ func buildLabelsAndAnnotations(ctrLabels map[string]string) (map[string]string,
return labels, annotations return labels, annotations
} }
func ListNamespaces(client *containerd.Client) []string {
set := []string{}
store := client.NamespaceService()
namespaces, err := store.List(context.Background())
if err != nil {
log.Printf("Error listing namespaces: %s", err.Error())
set = append(set, faasd.DefaultFunctionNamespace)
return set
}
for _, namespace := range namespaces {
labels, err := store.Labels(context.Background(), namespace)
if err != nil {
log.Printf("Error listing label for namespace %s: %s", namespace, err.Error())
continue
}
if _, found := labels[pkg.NamespaceLabel]; found {
set = append(set, namespace)
}
if !findNamespace(faasd.DefaultFunctionNamespace, set) {
set = append(set, faasd.DefaultFunctionNamespace)
}
}
return set
}
func findNamespace(target string, items []string) bool {
for _, n := range items {
if n == target {
return true
}
}
return false
}
func readMemoryLimitFromSpec(spec *specs.Spec) int64 {
if spec.Linux == nil || spec.Linux.Resources == nil || spec.Linux.Resources.Memory == nil || spec.Linux.Resources.Memory.Limit == nil {
return 0
}
return *spec.Linux.Resources.Memory.Limit
}

View File

@ -2,9 +2,10 @@ package handlers
import ( import (
"fmt" "fmt"
"github.com/opencontainers/runtime-spec/specs-go"
"reflect" "reflect"
"testing" "testing"
"github.com/opencontainers/runtime-spec/specs-go"
) )
func Test_BuildLabelsAndAnnotationsFromServiceSpec_Annotations(t *testing.T) { func Test_BuildLabelsAndAnnotationsFromServiceSpec_Annotations(t *testing.T) {
@ -31,56 +32,102 @@ func Test_BuildLabelsAndAnnotationsFromServiceSpec_Annotations(t *testing.T) {
} }
func Test_SplitMountToSecrets(t *testing.T) { func Test_SplitMountToSecrets(t *testing.T) {
type test struct { type testCase struct {
Name string Name string
Input []specs.Mount Input []specs.Mount
Expected []string Want []string
} }
tests := []test{ tests := []testCase{
{Name: "No matching openfaas secrets", Input: []specs.Mount{{Destination: "/foo/"}}, Expected: []string{}}, {Name: "No matching openfaas secrets", Input: []specs.Mount{{Destination: "/foo/"}}, Want: []string{}},
{Name: "Nil mounts", Input: nil, Expected: []string{}}, {Name: "Nil mounts", Input: nil, Want: []string{}},
{Name: "No Mounts", Input: []specs.Mount{{Destination: "/foo/"}}, Expected: []string{}}, {Name: "No Mounts", Input: []specs.Mount{{Destination: "/foo/"}}, Want: []string{}},
{Name: "One Mounts IS secret", Input: []specs.Mount{{Destination: "/var/openfaas/secrets/secret1"}}, Expected: []string{"secret1"}}, {Name: "One Mounts IS secret", Input: []specs.Mount{{Destination: "/var/openfaas/secrets/secret1"}}, Want: []string{"secret1"}},
{Name: "Multiple Mounts 1 secret", Input: []specs.Mount{{Destination: "/var/openfaas/secrets/secret1"}, {Destination: "/some/other/path"}}, Expected: []string{"secret1"}}, {Name: "Multiple Mounts 1 secret", Input: []specs.Mount{{Destination: "/var/openfaas/secrets/secret1"}, {Destination: "/some/other/path"}}, Want: []string{"secret1"}},
{Name: "Multiple Mounts all secrets", Input: []specs.Mount{{Destination: "/var/openfaas/secrets/secret1"}, {Destination: "/var/openfaas/secrets/secret2"}}, Expected: []string{"secret1", "secret2"}}, {Name: "Multiple Mounts all secrets", Input: []specs.Mount{{Destination: "/var/openfaas/secrets/secret1"}, {Destination: "/var/openfaas/secrets/secret2"}}, Want: []string{"secret1", "secret2"}},
} }
for _, tc := range tests { for _, tc := range tests {
t.Run(tc.Name, func(t *testing.T) { t.Run(tc.Name, func(t *testing.T) {
got := readSecretsFromMounts(tc.Input) got := readSecretsFromMounts(tc.Input)
if !reflect.DeepEqual(got, tc.Expected) { if !reflect.DeepEqual(got, tc.Want) {
t.Fatalf("expected %s, got %s", tc.Expected, got) t.Fatalf("Want %s, got %s", tc.Want, got)
} }
}) })
} }
} }
func Test_ProcessEnvToEnvVars(t *testing.T) { func Test_ProcessEnvToEnvVars(t *testing.T) {
type test struct { type testCase struct {
Name string Name string
Input []string Input []string
Expected map[string]string Want map[string]string
fprocess string fprocess string
} }
tests := []test{ tests := []testCase{
{Name: "No matching EnvVars", Input: []string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "fprocess=python index.py"}, Expected: make(map[string]string), fprocess: "python index.py"}, {Name: "No matching EnvVars", Input: []string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "fprocess=python index.py"}, Want: make(map[string]string), fprocess: "python index.py"},
{Name: "No EnvVars", Input: []string{}, Expected: make(map[string]string), fprocess: ""}, {Name: "No EnvVars", Input: []string{}, Want: make(map[string]string), fprocess: ""},
{Name: "One EnvVar", Input: []string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "fprocess=python index.py", "env=this"}, Expected: map[string]string{"env": "this"}, fprocess: "python index.py"}, {Name: "One EnvVar", Input: []string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "fprocess=python index.py", "env=this"}, Want: map[string]string{"env": "this"}, fprocess: "python index.py"},
{Name: "Multiple EnvVars", Input: []string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "this=that", "env=var", "fprocess=python index.py"}, Expected: map[string]string{"this": "that", "env": "var"}, fprocess: "python index.py"}, {Name: "Multiple EnvVars", Input: []string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "this=that", "env=var", "fprocess=python index.py"}, Want: map[string]string{"this": "that", "env": "var"}, fprocess: "python index.py"},
{Name: "Nil EnvVars", Input: nil, Expected: make(map[string]string)}, {Name: "Nil EnvVars", Input: nil, Want: make(map[string]string)},
} }
for _, tc := range tests { for _, tc := range tests {
t.Run(tc.Name, func(t *testing.T) { t.Run(tc.Name, func(t *testing.T) {
got, fprocess := readEnvFromProcessEnv(tc.Input) got, fprocess := readEnvFromProcessEnv(tc.Input)
if !reflect.DeepEqual(got, tc.Expected) { if !reflect.DeepEqual(got, tc.Want) {
t.Fatalf("expected: %s, got: %s", tc.Expected, got) t.Fatalf("Want: %s, got: %s", tc.Want, got)
} }
if fprocess != tc.fprocess { if fprocess != tc.fprocess {
t.Fatalf("expected fprocess: %s, got: %s", tc.fprocess, got) t.Fatalf("Want fprocess: %s, got: %s", tc.fprocess, got)
} }
}) })
} }
} }
func Test_findNamespace(t *testing.T) {
type testCase struct {
Name string
foundNamespaces []string
namespace string
Want bool
}
tests := []testCase{
{Name: "Namespace Found", namespace: "fn", foundNamespaces: []string{"fn", "openfaas-fn"}, Want: true},
{Name: "namespace Not Found", namespace: "fn", foundNamespaces: []string{"openfaas-fn"}, Want: false},
}
for _, tc := range tests {
t.Run(tc.Name, func(t *testing.T) {
got := findNamespace(tc.namespace, tc.foundNamespaces)
if got != tc.Want {
t.Fatalf("Want %t, got %t", tc.Want, got)
}
})
}
}
func Test_readMemoryLimitFromSpec(t *testing.T) {
type testCase struct {
Name string
Spec *specs.Spec
Want int64
}
testLimit := int64(64)
tests := []testCase{
{Name: "specs.Linux not found", Spec: &specs.Spec{Linux: nil}, Want: int64(0)},
{Name: "specs.LinuxResource not found", Spec: &specs.Spec{Linux: &specs.Linux{Resources: nil}}, Want: int64(0)},
{Name: "specs.LinuxMemory not found", Spec: &specs.Spec{Linux: &specs.Linux{Resources: &specs.LinuxResources{Memory: nil}}}, Want: int64(0)},
{Name: "specs.LinuxMemory.Limit not found", Spec: &specs.Spec{Linux: &specs.Linux{Resources: &specs.LinuxResources{Memory: &specs.LinuxMemory{Limit: nil}}}}, Want: int64(0)},
{Name: "Memory limit set as Want", Spec: &specs.Spec{Linux: &specs.Linux{Resources: &specs.LinuxResources{Memory: &specs.LinuxMemory{Limit: &testLimit}}}}, Want: int64(64)},
}
for _, tc := range tests {
t.Run(tc.Name, func(t *testing.T) {
got := readMemoryLimitFromSpec(tc.Spec)
if got != tc.Want {
t.Fatalf("Want %d, got %d", tc.Want, got)
}
})
}
}

View File

@ -4,8 +4,10 @@ import (
"fmt" "fmt"
"log" "log"
"net/url" "net/url"
"strings"
"github.com/containerd/containerd" "github.com/containerd/containerd"
faasd "github.com/openfaas/faasd/pkg"
) )
const watchdogPort = 8080 const watchdogPort = 8080
@ -19,11 +21,18 @@ func NewInvokeResolver(client *containerd.Client) *InvokeResolver {
} }
func (i *InvokeResolver) Resolve(functionName string) (url.URL, error) { func (i *InvokeResolver) Resolve(functionName string) (url.URL, error) {
log.Printf("Resolve: %q\n", functionName) actualFunctionName := functionName
log.Printf("Resolve: %q\n", actualFunctionName)
function, err := GetFunction(i.client, functionName) namespace := getNamespace(functionName, faasd.DefaultFunctionNamespace)
if strings.Contains(functionName, ".") {
actualFunctionName = strings.TrimSuffix(functionName, "."+namespace)
}
function, err := GetFunction(i.client, actualFunctionName, namespace)
if err != nil { if err != nil {
return url.URL{}, fmt.Errorf("%s not found", functionName) return url.URL{}, fmt.Errorf("%s not found", actualFunctionName)
} }
serviceIP := function.IP serviceIP := function.IP
@ -37,3 +46,11 @@ func (i *InvokeResolver) Resolve(functionName string) (url.URL, error) {
return *urlRes, nil return *urlRes, nil
} }
func getNamespace(name, defaultNamespace string) string {
namespace := defaultNamespace
if strings.Contains(name, ".") {
namespace = name[strings.LastIndexAny(name, ".")+1:]
}
return namespace
}

View File

@ -0,0 +1,18 @@
package handlers
import (
"encoding/json"
"net/http"
"github.com/containerd/containerd"
)
func MakeNamespacesLister(client *containerd.Client) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
list := ListNamespaces(client)
body, _ := json.Marshal(list)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
w.Write(body)
}
}

View File

@ -2,6 +2,7 @@ package handlers
import ( import (
"encoding/json" "encoding/json"
"k8s.io/apimachinery/pkg/api/resource"
"log" "log"
"net/http" "net/http"
@ -13,8 +14,21 @@ func MakeReadHandler(client *containerd.Client) func(w http.ResponseWriter, r *h
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
lookupNamespace := getRequestNamespace(readNamespaceFromQuery(r))
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client, lookupNamespace)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
res := []types.FunctionStatus{} res := []types.FunctionStatus{}
fns, err := ListFunctions(client) fns, err := ListFunctions(client, lookupNamespace)
if err != nil { if err != nil {
log.Printf("[Read] error listing functions. Error: %s\n", err) log.Printf("[Read] error listing functions. Error: %s\n", err)
http.Error(w, err.Error(), http.StatusBadRequest) http.Error(w, err.Error(), http.StatusBadRequest)
@ -24,6 +38,7 @@ func MakeReadHandler(client *containerd.Client) func(w http.ResponseWriter, r *h
for _, fn := range fns { for _, fn := range fns {
annotations := &fn.annotations annotations := &fn.annotations
labels := &fn.labels labels := &fn.labels
memory := resource.NewQuantity(fn.memoryLimit, resource.BinarySI)
res = append(res, types.FunctionStatus{ res = append(res, types.FunctionStatus{
Name: fn.name, Name: fn.name,
Image: fn.image, Image: fn.image,
@ -34,6 +49,7 @@ func MakeReadHandler(client *containerd.Client) func(w http.ResponseWriter, r *h
Secrets: fn.secrets, Secrets: fn.secrets,
EnvVars: fn.envVars, EnvVars: fn.envVars,
EnvProcess: fn.envProcess, EnvProcess: fn.envProcess,
Limits: &types.FunctionResources{Memory: memory.String()},
CreatedAt: fn.createdAt, CreatedAt: fn.createdAt,
}) })
} }

View File

@ -14,8 +14,21 @@ func MakeReplicaReaderHandler(client *containerd.Client) func(w http.ResponseWri
return func(w http.ResponseWriter, r *http.Request) { return func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r) vars := mux.Vars(r)
functionName := vars["name"] functionName := vars["name"]
lookupNamespace := getRequestNamespace(readNamespaceFromQuery(r))
if f, err := GetFunction(client, functionName); err == nil { // Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client, lookupNamespace)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
if f, err := GetFunction(client, functionName, lookupNamespace); err == nil {
found := types.FunctionStatus{ found := types.FunctionStatus{
Name: functionName, Name: functionName,
AvailableReplicas: uint64(f.replicas), AvailableReplicas: uint64(f.replicas),

View File

@ -13,7 +13,6 @@ import (
gocni "github.com/containerd/go-cni" gocni "github.com/containerd/go-cni"
"github.com/openfaas/faas-provider/types" "github.com/openfaas/faas-provider/types"
faasd "github.com/openfaas/faasd/pkg"
) )
func MakeReplicaUpdateHandler(client *containerd.Client, cni gocni.CNI) func(w http.ResponseWriter, r *http.Request) { func MakeReplicaUpdateHandler(client *containerd.Client, cni gocni.CNI) func(w http.ResponseWriter, r *http.Request) {
@ -40,16 +39,30 @@ func MakeReplicaUpdateHandler(client *containerd.Client, cni gocni.CNI) func(w h
return return
} }
namespace := getRequestNamespace(readNamespaceFromQuery(r))
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client, namespace)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
name := req.ServiceName name := req.ServiceName
if _, err := GetFunction(client, name); err != nil { if _, err := GetFunction(client, name, namespace); err != nil {
msg := fmt.Sprintf("service %s not found", name) msg := fmt.Sprintf("service %s not found", name)
log.Printf("[Scale] %s\n", msg) log.Printf("[Scale] %s\n", msg)
http.Error(w, msg, http.StatusNotFound) http.Error(w, msg, http.StatusNotFound)
return return
} }
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace) ctx := namespaces.WithNamespace(context.Background(), namespace)
ctr, ctrErr := client.LoadContainer(ctx, name) ctr, ctrErr := client.LoadContainer(ctx, name)
if ctrErr != nil { if ctrErr != nil {

View File

@ -15,6 +15,7 @@ import (
) )
const secretFilePermission = 0644 const secretFilePermission = 0644
const secretDirPermission = 0755
func MakeSecretHandler(c *containerd.Client, mountPath string) func(w http.ResponseWriter, r *http.Request) { func MakeSecretHandler(c *containerd.Client, mountPath string) func(w http.ResponseWriter, r *http.Request) {
@ -46,6 +47,22 @@ func MakeSecretHandler(c *containerd.Client, mountPath string) func(w http.Respo
} }
func listSecrets(c *containerd.Client, w http.ResponseWriter, r *http.Request, mountPath string) { func listSecrets(c *containerd.Client, w http.ResponseWriter, r *http.Request, mountPath string) {
lookupNamespace := getRequestNamespace(readNamespaceFromQuery(r))
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(c, lookupNamespace)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
mountPath = getNamespaceSecretMountPath(mountPath, lookupNamespace)
files, err := ioutil.ReadDir(mountPath) files, err := ioutil.ReadDir(mountPath)
if err != nil { if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError) http.Error(w, err.Error(), http.StatusInternalServerError)
@ -54,7 +71,7 @@ func listSecrets(c *containerd.Client, w http.ResponseWriter, r *http.Request, m
secrets := []types.Secret{} secrets := []types.Secret{}
for _, f := range files { for _, f := range files {
secrets = append(secrets, types.Secret{Name: f.Name()}) secrets = append(secrets, types.Secret{Name: f.Name(), Namespace: lookupNamespace})
} }
bytesOut, _ := json.Marshal(secrets) bytesOut, _ := json.Marshal(secrets)
@ -69,7 +86,30 @@ func createSecret(c *containerd.Client, w http.ResponseWriter, r *http.Request,
return return
} }
err = ioutil.WriteFile(path.Join(mountPath, secret.Name), []byte(secret.Value), secretFilePermission) err = validateSecret(secret)
if err != nil {
log.Printf("[secret] error %s", err.Error())
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
log.Printf("[secret] is valid: %q", secret.Name)
namespace := getRequestNamespace(secret.Namespace)
mountPath = getNamespaceSecretMountPath(mountPath, namespace)
err = os.MkdirAll(mountPath, secretDirPermission)
if err != nil {
log.Printf("[secret] error %s", err.Error())
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
data := secret.RawValue
if len(data) == 0 {
data = []byte(secret.Value)
}
err = ioutil.WriteFile(path.Join(mountPath, secret.Name), data, secretFilePermission)
if err != nil { if err != nil {
log.Printf("[secret] error %s", err.Error()) log.Printf("[secret] error %s", err.Error())
@ -86,6 +126,9 @@ func deleteSecret(c *containerd.Client, w http.ResponseWriter, r *http.Request,
return return
} }
namespace := getRequestNamespace(readNamespaceFromQuery(r))
mountPath = getNamespaceSecretMountPath(mountPath, namespace)
err = os.Remove(path.Join(mountPath, secret.Name)) err = os.Remove(path.Join(mountPath, secret.Name))
if err != nil { if err != nil {
@ -107,10 +150,6 @@ func parseSecret(r *http.Request) (types.Secret, error) {
return secret, err return secret, err
} }
if isTraversal(secret.Name) {
return secret, fmt.Errorf(traverseErrorSt)
}
return secret, err return secret, err
} }
@ -120,3 +159,13 @@ func isTraversal(name string) bool {
return strings.Contains(name, fmt.Sprintf("%s", string(os.PathSeparator))) || return strings.Contains(name, fmt.Sprintf("%s", string(os.PathSeparator))) ||
strings.Contains(name, "..") strings.Contains(name, "..")
} }
func validateSecret(secret types.Secret) error {
if strings.TrimSpace(secret.Name) == "" {
return fmt.Errorf("non-empty name is required")
}
if isTraversal(secret.Name) {
return fmt.Errorf(traverseErrorSt)
}
return nil
}

View File

@ -1,63 +1,163 @@
package handlers package handlers
import ( import (
"bytes"
"encoding/json"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os"
"path/filepath"
"reflect"
"strings"
"testing" "testing"
"github.com/openfaas/faas-provider/types" "github.com/openfaas/faas-provider/types"
) )
func Test_parseSecretValidName(t *testing.T) { func Test_parseSecret(t *testing.T) {
cases := []struct {
name string
payload string
expError string
expSecret types.Secret
}{
{
name: "no error when name is valid without extention and with no traversal",
payload: `{"name": "authorized_keys", "value": "foo"}`,
expSecret: types.Secret{Name: "authorized_keys", Value: "foo"},
},
{
name: "no error when name is valid and parses RawValue correctly",
payload: `{"name": "authorized_keys", "rawValue": "YmFy"}`,
expSecret: types.Secret{Name: "authorized_keys", RawValue: []byte("bar")},
},
{
name: "no error when name is valid with dot and with no traversal",
payload: `{"name": "authorized.keys", "value": "foo"}`,
expSecret: types.Secret{Name: "authorized.keys", Value: "foo"},
},
}
s := types.Secret{Name: "authorized_keys"} for _, tc := range cases {
body, _ := json.Marshal(s) t.Run(tc.name, func(t *testing.T) {
reader := bytes.NewReader(body) reader := strings.NewReader(tc.payload)
r := httptest.NewRequest(http.MethodPost, "/", reader) r := httptest.NewRequest(http.MethodPost, "/", reader)
_, err := parseSecret(r) secret, err := parseSecret(r)
if err != nil && tc.expError == "" {
t.Fatalf("unexpected error: %s", err)
return
}
if tc.expError != "" {
if err == nil {
t.Fatalf("expected error: %s, got nil", tc.expError)
}
if err.Error() != tc.expError {
t.Fatalf("expected error: %s, got: %s", tc.expError, err)
}
return
}
if !reflect.DeepEqual(secret, tc.expSecret) {
t.Fatalf("expected secret: %+v, got: %+v", tc.expSecret, secret)
}
})
}
}
func TestSecretCreation(t *testing.T) {
mountPath, err := os.MkdirTemp("", "test_secret_creation")
if err != nil { if err != nil {
t.Fatalf("secret name is valid with no traversal characters") t.Fatalf("unexpected error while creating temp directory: %s", err)
} }
}
defer os.RemoveAll(mountPath)
func Test_parseSecretValidNameWithDot(t *testing.T) {
handler := MakeSecretHandler(nil, mountPath)
s := types.Secret{Name: "authorized.keys"}
body, _ := json.Marshal(s) cases := []struct {
reader := bytes.NewReader(body) name string
r := httptest.NewRequest(http.MethodPost, "/", reader) verb string
_, err := parseSecret(r) payload string
status int
if err != nil { secretPath string
t.Fatalf("secret name is valid with no traversal characters") secret string
} err string
} }{
{
func Test_parseSecretWithTraversalWithSlash(t *testing.T) { name: "returns error when the name contains a traversal",
verb: http.MethodPost,
s := types.Secret{Name: "/root/.ssh/authorized_keys"} payload: `{"name": "/root/.ssh/authorized_keys", "value": "foo"}`,
body, _ := json.Marshal(s) status: http.StatusBadRequest,
reader := bytes.NewReader(body) err: "directory traversal found in name\n",
r := httptest.NewRequest(http.MethodPost, "/", reader) },
_, err := parseSecret(r) {
name: "returns error when the name contains a traversal",
if err == nil { verb: http.MethodPost,
t.Fatalf("secret name should fail due to path traversal") payload: `{"name": "..", "value": "foo"}`,
} status: http.StatusBadRequest,
} err: "directory traversal found in name\n",
},
func Test_parseSecretWithTraversalWithDoubleDot(t *testing.T) { {
name: "empty request returns a validation error",
s := types.Secret{Name: ".."} verb: http.MethodPost,
body, _ := json.Marshal(s) payload: `{}`,
reader := bytes.NewReader(body) status: http.StatusBadRequest,
r := httptest.NewRequest(http.MethodPost, "/", reader) err: "non-empty name is required\n",
_, err := parseSecret(r) },
{
if err == nil { name: "can create secret from string",
t.Fatalf("secret name should fail due to path traversal") verb: http.MethodPost,
payload: `{"name": "foo", "value": "bar"}`,
status: http.StatusOK,
secretPath: "/openfaas-fn/foo",
secret: "bar",
},
{
name: "can create secret from raw value",
verb: http.MethodPost,
payload: `{"name": "foo", "rawValue": "YmFy"}`,
status: http.StatusOK,
secretPath: "/openfaas-fn/foo",
secret: "bar",
},
{
name: "can create secret in non-default namespace from raw value",
verb: http.MethodPost,
payload: `{"name": "pity", "rawValue": "dGhlIGZvbw==", "namespace": "a-team"}`,
status: http.StatusOK,
secretPath: "/a-team/pity",
secret: "the foo",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
req := httptest.NewRequest(tc.verb, "http://example.com/foo", strings.NewReader(tc.payload))
w := httptest.NewRecorder()
handler(w, req)
resp := w.Result()
if resp.StatusCode != tc.status {
t.Logf("response body: %s", w.Body.String())
t.Fatalf("expected status: %d, got: %d", tc.status, resp.StatusCode)
}
if resp.StatusCode != http.StatusOK && w.Body.String() != tc.err {
t.Fatalf("expected error message: %q, got %q", tc.err, w.Body.String())
}
if tc.secretPath != "" {
data, err := os.ReadFile(filepath.Join(mountPath, tc.secretPath))
if err != nil {
t.Fatalf("can not read the secret from disk: %s", err)
}
if string(data) != tc.secret {
t.Fatalf("expected secret value: %s, got %s", tc.secret, string(data))
}
}
})
} }
} }

View File

@ -13,7 +13,6 @@ import (
gocni "github.com/containerd/go-cni" gocni "github.com/containerd/go-cni"
"github.com/openfaas/faas-provider/types" "github.com/openfaas/faas-provider/types"
faasd "github.com/openfaas/faasd/pkg"
"github.com/openfaas/faasd/pkg/cninetwork" "github.com/openfaas/faasd/pkg/cninetwork"
"github.com/openfaas/faasd/pkg/service" "github.com/openfaas/faasd/pkg/service"
) )
@ -41,8 +40,23 @@ func MakeUpdateHandler(client *containerd.Client, cni gocni.CNI, secretMountPath
return return
} }
name := req.Service name := req.Service
namespace := getRequestNamespace(req.Namespace)
function, err := GetFunction(client, name) // Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client, namespace)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
namespaceSecretMountPath := getNamespaceSecretMountPath(secretMountPath, namespace)
function, err := GetFunction(client, name, namespace)
if err != nil { if err != nil {
msg := fmt.Sprintf("service %s not found", name) msg := fmt.Sprintf("service %s not found", name)
log.Printf("[Update] %s\n", msg) log.Printf("[Update] %s\n", msg)
@ -50,12 +64,12 @@ func MakeUpdateHandler(client *containerd.Client, cni gocni.CNI, secretMountPath
return return
} }
err = validateSecrets(secretMountPath, req.Secrets) err = validateSecrets(namespaceSecretMountPath, req.Secrets)
if err != nil { if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest) http.Error(w, err.Error(), http.StatusBadRequest)
} }
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace) ctx := namespaces.WithNamespace(context.Background(), namespace)
if _, err := prepull(ctx, req, client, alwaysPull); err != nil { if _, err := prepull(ctx, req, client, alwaysPull); err != nil {
log.Printf("[Update] error with pre-pull: %s, %s\n", name, err) log.Printf("[Update] error with pre-pull: %s, %s\n", name, err)
@ -78,7 +92,7 @@ func MakeUpdateHandler(client *containerd.Client, cni gocni.CNI, secretMountPath
// The pull has already been done in prepull, so we can force this pull to "false" // The pull has already been done in prepull, so we can force this pull to "false"
pull := false pull := false
if err := deploy(ctx, req, client, cni, secretMountPath, pull); err != nil { if err := deploy(ctx, req, client, cni, namespaceSecretMountPath, pull); err != nil {
log.Printf("[Update] error deploying %s, error: %s\n", name, err) log.Printf("[Update] error deploying %s, error: %s\n", name, err)
http.Error(w, err.Error(), http.StatusBadRequest) http.Error(w, err.Error(), http.StatusBadRequest)
return return

View File

@ -0,0 +1,49 @@
package handlers
import (
"context"
"net/http"
"path"
"github.com/containerd/containerd"
"github.com/openfaas/faasd/pkg"
faasd "github.com/openfaas/faasd/pkg"
)
func getRequestNamespace(namespace string) string {
if len(namespace) > 0 {
return namespace
}
return faasd.DefaultFunctionNamespace
}
func readNamespaceFromQuery(r *http.Request) string {
q := r.URL.Query()
return q.Get("namespace")
}
func getNamespaceSecretMountPath(userSecretPath string, namespace string) string {
return path.Join(userSecretPath, namespace)
}
// validNamespace indicates whether the namespace is eligable to be
// used for OpenFaaS functions.
func validNamespace(client *containerd.Client, namespace string) (bool, error) {
if namespace == faasd.DefaultFunctionNamespace {
return true, nil
}
store := client.NamespaceService()
labels, err := store.Labels(context.Background(), namespace)
if err != nil {
return false, err
}
if value, found := labels[pkg.NamespaceLabel]; found && value == "true" {
return true, nil
}
return false, nil
}

View File

@ -0,0 +1,75 @@
package handlers
import (
"fmt"
"net/http"
"net/http/httptest"
"testing"
faasd "github.com/openfaas/faasd/pkg"
)
func Test_getRequestNamespace(t *testing.T) {
tables := []struct {
name string
requestNamespace string
expectedNamespace string
}{
{name: "RequestNamespace is not provided", requestNamespace: "", expectedNamespace: faasd.DefaultFunctionNamespace},
{name: "RequestNamespace is provided", requestNamespace: "user-namespace", expectedNamespace: "user-namespace"},
}
for _, tc := range tables {
t.Run(tc.name, func(t *testing.T) {
actualNamespace := getRequestNamespace(tc.requestNamespace)
if actualNamespace != tc.expectedNamespace {
t.Errorf("Want: %s, got: %s", actualNamespace, tc.expectedNamespace)
}
})
}
}
func Test_getNamespaceSecretMountPath(t *testing.T) {
userSecretPath := "/var/openfaas/secrets"
tables := []struct {
name string
requestNamespace string
expectedSecretPath string
}{
{name: "Default Namespace is provided", requestNamespace: faasd.DefaultFunctionNamespace, expectedSecretPath: "/var/openfaas/secrets/" + faasd.DefaultFunctionNamespace},
{name: "User Namespace is provided", requestNamespace: "user-namespace", expectedSecretPath: "/var/openfaas/secrets/user-namespace"},
}
for _, tc := range tables {
t.Run(tc.name, func(t *testing.T) {
actualNamespace := getNamespaceSecretMountPath(userSecretPath, tc.requestNamespace)
if actualNamespace != tc.expectedSecretPath {
t.Errorf("Want: %s, got: %s", actualNamespace, tc.expectedSecretPath)
}
})
}
}
func Test_readNamespaceFromQuery(t *testing.T) {
tables := []struct {
name string
queryNamespace string
expectedNamespace string
}{
{name: "No Namespace is provided", queryNamespace: "", expectedNamespace: ""},
{name: "User Namespace is provided", queryNamespace: "user-namespace", expectedNamespace: "user-namespace"},
}
for _, tc := range tables {
t.Run(tc.name, func(t *testing.T) {
url := fmt.Sprintf("/test?namespace=%s", tc.queryNamespace)
r := httptest.NewRequest(http.MethodGet, url, nil)
actualNamespace := readNamespaceFromQuery(r)
if actualNamespace != tc.expectedNamespace {
t.Errorf("Want: %s, got: %s", actualNamespace, tc.expectedNamespace)
}
})
}
}

View File

@ -33,7 +33,7 @@ func Remove(ctx context.Context, client *containerd.Client, name string) error {
if errdefs.IsNotFound(err) { if errdefs.IsNotFound(err) {
taskFound = false taskFound = false
} else { } else {
return fmt.Errorf("unable to get task %s: ", err) return fmt.Errorf("unable to get task %w: ", err)
} }
} }
@ -47,12 +47,12 @@ func Remove(ctx context.Context, client *containerd.Client, name string) error {
log.Printf("Need to kill task: %s\n", name) log.Printf("Need to kill task: %s\n", name)
if err = killTask(ctx, t); err != nil { if err = killTask(ctx, t); err != nil {
return fmt.Errorf("error killing task %s, %s, %s", container.ID(), name, err) return fmt.Errorf("error killing task %s, %s, %w", container.ID(), name, err)
} }
} }
if err := container.Delete(ctx, containerd.WithSnapshotCleanup); err != nil { if err := container.Delete(ctx, containerd.WithSnapshotCleanup); err != nil {
return fmt.Errorf("error deleting container %s, %s, %s", container.ID(), name, err) return fmt.Errorf("error deleting container %s, %s, %w", container.ID(), name, err)
} }
} else { } else {
@ -79,9 +79,10 @@ func killTask(ctx context.Context, task containerd.Task) error {
if task != nil { if task != nil {
wait, err := task.Wait(ctx) wait, err := task.Wait(ctx)
if err != nil { if err != nil {
err = fmt.Errorf("error waiting on task: %s", err) log.Printf("error waiting on task: %s", err)
return return
} }
if err := task.Kill(ctx, unix.SIGTERM, containerd.WithKillAll); err != nil { if err := task.Kill(ctx, unix.SIGTERM, containerd.WithKillAll); err != nil {
log.Printf("error killing container task: %s", err) log.Printf("error killing container task: %s", err)
} }

View File

@ -8,6 +8,8 @@ import (
"os" "os"
"path" "path"
"sort" "sort"
"strconv"
"strings"
"github.com/alexellis/k3sup/pkg/env" "github.com/alexellis/k3sup/pkg/env"
"github.com/compose-spec/compose-go/loader" "github.com/compose-spec/compose-go/loader"
@ -26,7 +28,8 @@ import (
) )
const ( const (
workingDirectoryPermission = 0644 // workingDirectoryPermission user read/write/execute, group and others: read-only
workingDirectoryPermission = 0744
) )
type Service struct { type Service struct {
@ -79,7 +82,7 @@ func NewSupervisor(sock string) (*Supervisor, error) {
} }
func (s *Supervisor) Start(svcs []Service) error { func (s *Supervisor) Start(svcs []Service) error {
ctx := namespaces.WithNamespace(context.Background(), faasdNamespace) ctx := namespaces.WithNamespace(context.Background(), FaasdNamespace)
wd, _ := os.Getwd() wd, _ := os.Getwd()
@ -145,6 +148,28 @@ func (s *Supervisor) Start(svcs []Service) error {
Type: "bind", Type: "bind",
Options: []string{"rbind", "rw"}, Options: []string{"rbind", "rw"},
}) })
// Only create directories, not files.
// Some files don't have a suffix, such as secrets.
if len(path.Ext(mnt.Src)) == 0 &&
!strings.HasPrefix(mnt.Src, "/var/lib/faasd/secrets/") {
// src is already prefixed with wd from an earlier step
src := mnt.Src
fmt.Printf("Creating local directory: %s\n", src)
if err := os.MkdirAll(src, workingDirectoryPermission); err != nil {
if !errors.Is(os.ErrExist, err) {
fmt.Printf("Unable to create: %s, %s\n", src, err)
}
}
if len(svc.User) > 0 {
uid, err := strconv.Atoi(svc.User)
if err == nil {
if err := os.Chown(src, uid, -1); err != nil {
fmt.Printf("Unable to chown: %s to %d, error: %s\n", src, uid, err)
}
}
}
}
} }
} }
@ -243,7 +268,7 @@ func (s *Supervisor) Close() {
} }
func (s *Supervisor) Remove(svcs []Service) error { func (s *Supervisor) Remove(svcs []Service) error {
ctx := namespaces.WithNamespace(context.Background(), faasdNamespace) ctx := namespaces.WithNamespace(context.Background(), FaasdNamespace)
for _, svc := range svcs { for _, svc := range svcs {
err := cninetwork.DeleteCNINetwork(ctx, s.cni, s.client, svc.Name) err := cninetwork.DeleteCNINetwork(ctx, s.cni, s.client, svc.Name)