Compare commits

..

1 Commits

Author SHA1 Message Date
97c11b3f5d Proxy the gateway using TCP
There appeared to be an issue with logs appearing #98 and #68

@LucasRoesler spent a considerable amount of time looking into
this and concluded that the faas-provider and approach we are
taking to stream logs from journalctl as a process was
working as expected.

The issue appears to have been with the proxy code and its
use of a HTTP connection. Somewhere within the code, a buffer
was holding onto the data before flushing it 20-30 seconds later

This appeared to users as if the logs were not working at all.

Before fixing, the gateway container was tested by exposing
it over an SSH tunnel and inlets tunnel, both worked as
expected. The updates have been tested on multipass with
Ubuntu 18.04 and a binary built locally.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-09-08 11:33:11 +01:00
2640 changed files with 351308 additions and 395991 deletions

View File

@ -1,37 +1,16 @@
## Due diligence
<!-- Due dilligence -->
## My actions before raising this issue
Before you ask for help or support, make sure that you've [consulted the manual for faasd](https://openfaas.gumroad.com/l/serverless-for-everyone-else). We can't answer questions that are already covered by the manual.
<!-- How is this affecting you? What task are you trying to accomplish? -->
## Why do you need this?
<!-- Attempts to mask or hide this may result in the issue being closed -->
## Who is this for?
What company is this for? Are you listed in the [ADOPTERS.md](https://github.com/openfaas/faas/blob/master/ADOPTERS.md) file?
<!--- Provide a general summary of the issue in the Title above -->
## Expected Behaviour
<!--- If you're describing a bug, tell us what should happen -->
<!--- If you're suggesting a change/improvement, tell us how it should work -->
## Current Behaviour
<!--- If describing a bug, tell us what happens instead of the expected behavior -->
<!--- If suggesting a change/improvement, explain the difference from current behavior -->
## List All Possible Solutions and Workarounds
<!--- Suggest a fix/reason for the bug, or ideas how to implement -->
<!--- the addition or change -->
<!--- Is there a workaround which could avoid making changes? -->
## Which Solution Do You Recommend?
<!--- Pick your preferred solution, if you were to implement and maintain this change -->
## Possible Solution
<!--- Not obligatory, but suggest a fix/reason for the bug, -->
<!--- or ideas how to implement the addition or change -->
## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
@ -41,6 +20,10 @@ What company is this for? Are you listed in the [ADOPTERS.md](https://github.com
3.
4.
## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
## Your Environment
* OS and architecture:
@ -55,6 +38,4 @@ containerd -version
uname -a
cat /etc/os-release
faasd version
```

View File

@ -1,32 +0,0 @@
name: build
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
env:
GO111MODULE: off
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
with:
fetch-depth: 1
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: 1.22.x
- name: test
run: make test
- name: dist
run: make dist
- name: prepare-test
run: make prepare-test
- name: test e2e
run: make test-e2e

View File

@ -1,26 +0,0 @@
name: publish
on:
push:
tags:
- '*'
jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
with:
fetch-depth: 1
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: 1.22.x
- name: Make publish
run: make publish
- name: Upload release binaries
uses: alexellis/upload-assets@0.4.1
env:
GITHUB_TOKEN: ${{ github.token }}
with:
asset_paths: '["./bin/faasd*"]'

View File

@ -1,18 +0,0 @@
name: Verify Docker Compose Images
on:
push:
paths:
- '**.yaml'
jobs:
verifyImages:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
- uses: alexellis/setup-arkade@v3
- name: Verify chart images
id: verify_images
run: |
VERBOSE=true make verify-compose

31
.travis.yml Normal file
View File

@ -0,0 +1,31 @@
sudo: required
language: go
go:
- '1.13'
addons:
apt:
packages:
- runc
script:
- make test
- make dist
- make prepare-test
- make test-e2e
deploy:
provider: releases
api_key:
secure: bccOSB+Mbk5ZJHyJfX82Xg/3/7mxiAYHx7P5m5KS1ncDuRpJBFjDV8Nx2PWYg341b5SMlCwsS3IJ9NkoGvRSKK+3YqeNfTeMabVNdKC2oL1i+4pdxGlbl57QXkzT4smqE8AykZEo4Ujk42rEr3e0gSHT2rXkV+Xt0xnoRVXn2tSRUDwsmwANnaBj6KpH2SjJ/lsfTifxrRB65uwcePaSjkqwR6htFraQtpONC9xYDdek6EoVQmoft/ONZJqi7HR+OcA1yhSt93XU6Vaf3678uLlPX9c/DxgIU9UnXRaOd0UUEiTHaMMWDe/bJSrKmgL7qY05WwbGMsXO/RdswwO1+zwrasrwf86SjdGX/P9AwobTW3eTEiBqw2J77UVbvLzDDoyJ5KrkbHRfPX8aIPO4OG9eHy/e7C3XVx4qv9bJBXQ3qD9YJtei9jmm8F/MCdPWuVYC0hEvHtuhP/xMm4esNUjFM5JUfDucvAuLL34NBYHBDP2XNuV4DkgQQPakfnlvYBd7OqyXCU6pzyWSasXpD1Rz8mD/x8aTUl2Ya4bnXQ8qAa5cnxfPqN2ADRlTw1qS7hl6LsXzNQ6r1mbuh/uFi67ybElIjBTfuMEeJOyYHkkLUHIBpooKrPyr0luAbf0By2D2N/eQQnM/RpixHNfZG/mvXx8ZCrs+wxgvG1Rm7rM=
file:
- ./bin/faasd
- ./bin/faasd-armhf
- ./bin/faasd-arm64
skip_cleanup: true
on:
tags: true
env: GO111MODULE=off

63
EULA.md
View File

@ -1,63 +0,0 @@
## OpenFaaS Community Edition (CE) EULA Addendum
faasd is part of OpenFaaS Community Edition (CE) and is subject to the [OpenFaaS CE EULA](https://github.com/openfaas/faas/blob/master/EULA.md). Additional clauses apply to faasd only, to enable certain commercial usage for Individuals and within Small Business Environments.
End User License Agreement (EULA) Addendum for faasd Licensed Software.
This addendum to the OpenFaaS Community Edition EULA applies specifically to the use of faasd. It governs the conditions under which faasd may be used within Small Business Environments and for Personal Use.
1. Definitions
1.1 Small Business Environment:
A "Small Business Environment" is defined as any commercial entity with fewer than 50 employees and less than US $5,000,000 (or equivalent local currency) in annual revenue. Internal usage refers to faasd being used solely within the business for developing, testing, or deploying solutions for internal purposes or hosted services offered by the business, without distributing faasd itself to end clients or customers.
1.2 Personal Use:
"Personal Use" refers to the use of faasd by individuals for personal or commercial purposes, including hosting or running applications. This includes using faasd as part of a hosted solution (e.g., hosting functions or running a commercial application) but does not permit distributing faasd itself to clients or customers.
2. License Grant for faasd
2.1 Small Business Environment Use:
faasd may be used for limited commercial purposes within a Small Business Environment. This includes deploying faasd as part of a hosted solution or for internal usage (e.g., running functions or commercial applications for internal or customer-facing services), provided that faasd itself is not distributed, resold, or sublicensed to end clients or customers. The license covers internal use or as part of a hosted solution but restricts distributing faasd binaries or code to any third parties.
2.2 Personal Use:
faasd may be used by individuals for personal projects or commercial purposes, including hosting solutions for their own commercial applications or functions. This includes using faasd as part of a hosted solution (e.g., running functions or a commercial application within faasd), provided that faasd itself is not distributed to end clients or customers as part of the solution.
3. Restrictions on faasd Use
3.1 No Resale or Redistribution:
faasd may not be resold, distributed, or provided as part of a product or service directly to end clients or customers, regardless of whether those entities meet the Small Business Environment criteria outlined above. This includes embedding faasd into a software or service that is distributed to other businesses or individuals.
3.2 Non-transferability:
The rights to use faasd under this addendum are non-transferable. If the user or business does not meet the definition of a Small Business Environment or if Personal Use criteria do not apply, a separate commercial license must be obtained from OpenFaaS Ltd. If the entity meets the Small Business Environment or Personal Use criteria, no additional license is required.
4. Compliance
4.1 License Enforcement:
OpenFaaS Ltd reserves the right to implement reasonable measures to verify compliance with this faasd EULA addendum. The Customer agrees to cooperate with any compliance verification efforts conducted by OpenFaaS Ltd.
5. Termination of faasd License
This addendum and the license granted hereunder are effective upon installation and use of faasd. OpenFaaS Ltd reserves the right to terminate this license if any of the terms outlined in this addendum are violated. Upon termination, all copies of faasd must be deleted.
6. Limitation of Liability
6.1 Warranty Disclaimer:
faasd is provided "as is" without warranty of any kind. The user assumes all risks associated with using faasd.
6.2 Liability Limitations:
OpenFaaS Ltd shall not be liable for any indirect, special, incidental, or consequential damages arising out of the use or inability to use faasd.
7. Contact Information
For questions regarding this faasd EULA Addendum or to inquire about commercial licenses, please contact OpenFaaS Ltd at:
Email: contact@openfaas.com

691
Gopkg.lock generated Normal file
View File

@ -0,0 +1,691 @@
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
digest = "1:d5e752c67b445baa5b6cb6f8aa706775c2aa8e41aca95a0c651520ff2c80361a"
name = "github.com/Microsoft/go-winio"
packages = [
".",
"pkg/guid",
]
pruneopts = "UT"
revision = "6c72808b55902eae4c5943626030429ff20f3b63"
version = "v0.4.14"
[[projects]]
digest = "1:f06a14a8b60a7a9cdbf14ed52272faf4ff5de4ed7c784ff55b64995be98ac59f"
name = "github.com/Microsoft/hcsshim"
packages = [
".",
"internal/cow",
"internal/hcs",
"internal/hcserror",
"internal/hns",
"internal/interop",
"internal/log",
"internal/logfields",
"internal/longpath",
"internal/mergemaps",
"internal/oc",
"internal/safefile",
"internal/schema1",
"internal/schema2",
"internal/timeout",
"internal/vmcompute",
"internal/wclayer",
"osversion",
]
pruneopts = "UT"
revision = "9e921883ac929bbe515b39793ece99ce3a9d7706"
[[projects]]
digest = "1:d7086e6a64a9e4fa54aaf56ce42ead0be1300b0285604c4d306438880db946ad"
name = "github.com/alexellis/go-execute"
packages = ["pkg/v1"]
pruneopts = "UT"
revision = "8697e4e28c5e3ce441ff8b2b6073035606af2fe9"
version = "0.4.0"
[[projects]]
digest = "1:345f6fa182d72edfa3abc493881c3fa338a464d93b1e2169cda9c822fde31655"
name = "github.com/alexellis/k3sup"
packages = ["pkg/env"]
pruneopts = "UT"
revision = "629c0bc6b50f71ab93a1fbc8971a5bd05dc581eb"
version = "0.9.3"
[[projects]]
branch = "master"
digest = "1:cda177c07c87c648b1aaa37290717064a86d337a5dc6b317540426872d12de52"
name = "github.com/compose-spec/compose-go"
packages = [
"envfile",
"interpolation",
"loader",
"schema",
"template",
"types",
]
pruneopts = "UT"
revision = "36d8ce368e05d2ae83c86b2987f20f7c20d534a6"
[[projects]]
digest = "1:cf83a14c8042951b0dcd74758fc32258111ecc7838cbdf5007717172cab9ca9b"
name = "github.com/containerd/containerd"
packages = [
".",
"api/services/containers/v1",
"api/services/content/v1",
"api/services/diff/v1",
"api/services/events/v1",
"api/services/images/v1",
"api/services/introspection/v1",
"api/services/leases/v1",
"api/services/namespaces/v1",
"api/services/snapshots/v1",
"api/services/tasks/v1",
"api/services/version/v1",
"api/types",
"api/types/task",
"archive",
"archive/compression",
"cio",
"containers",
"content",
"content/proxy",
"defaults",
"diff",
"errdefs",
"events",
"events/exchange",
"filters",
"identifiers",
"images",
"images/archive",
"labels",
"leases",
"leases/proxy",
"log",
"mount",
"namespaces",
"oci",
"pkg/dialer",
"platforms",
"plugin",
"reference",
"remotes",
"remotes/docker",
"remotes/docker/schema1",
"rootfs",
"runtime/linux/runctypes",
"runtime/v2/logging",
"runtime/v2/runc/options",
"snapshots",
"snapshots/proxy",
"sys",
"version",
]
pruneopts = "UT"
revision = "ff48f57fc83a8c44cf4ad5d672424a98ba37ded6"
version = "v1.3.2"
[[projects]]
digest = "1:e4414857969cfbe45c7dab0a012aad4855bf7167c25d672a182cb18676424a0c"
name = "github.com/containerd/continuity"
packages = [
"fs",
"pathdriver",
"syscallx",
"sysx",
]
pruneopts = "UT"
revision = "f2a389ac0a02ce21c09edd7344677a601970f41c"
[[projects]]
digest = "1:1b9a7426259b5333d575785e21e1bd0decf18208f5bfb6424d24a50d5ddf83d0"
name = "github.com/containerd/fifo"
packages = ["."]
pruneopts = "UT"
revision = "bda0ff6ed73c67bfb5e62bc9c697f146b7fd7f13"
[[projects]]
branch = "master"
digest = "1:2301a9a859e3b0946e2ddd6961ba6faf6857e6e68bc9293db758dbe3b17cc35e"
name = "github.com/containerd/go-cni"
packages = ["."]
pruneopts = "UT"
revision = "c154a49e2c754b83ebfb12ebf1362213b94d23e6"
[[projects]]
digest = "1:6d66a41dbbc6819902f1589d0550bc01c18032c0a598a7cd656731e6df73861b"
name = "github.com/containerd/ttrpc"
packages = ["."]
pruneopts = "UT"
revision = "92c8520ef9f86600c650dd540266a007bf03670f"
[[projects]]
digest = "1:7b4683388adabc709dbb082c13ba35967f072379c85b4acde997c1ca75af5981"
name = "github.com/containerd/typeurl"
packages = ["."]
pruneopts = "UT"
revision = "a93fcdb778cd272c6e9b3028b2f42d813e785d40"
[[projects]]
digest = "1:1a07bbfee1d0534e8dda4773948e6dcd3a061ea7ab047ce04619476946226483"
name = "github.com/containernetworking/cni"
packages = [
"libcni",
"pkg/invoke",
"pkg/types",
"pkg/types/020",
"pkg/types/current",
"pkg/version",
]
pruneopts = "UT"
revision = "4cfb7b568922a3c79a23e438dc52fe537fc9687e"
version = "v0.7.1"
[[projects]]
digest = "1:bcf36df8d43860bfde913d008301aef27c6e9a303582118a837c4a34c0d18167"
name = "github.com/coreos/go-systemd"
packages = ["journal"]
pruneopts = "UT"
revision = "d3cd4ed1dbcf5835feba465b180436db54f20228"
version = "v21"
[[projects]]
digest = "1:92ebc9c068ab8e3fff03a58694ee33830964f6febd0130069aadce328802de14"
name = "github.com/docker/cli"
packages = [
"cli/config",
"cli/config/configfile",
"cli/config/credentials",
"cli/config/types",
]
pruneopts = "UT"
revision = "99c5edceb48d64c1aa5d09b8c9c499d431d98bb9"
version = "v19.03.5"
[[projects]]
digest = "1:e495f9f1fb2bae55daeb76e099292054fe1f734947274b3cfc403ccda595d55a"
name = "github.com/docker/distribution"
packages = [
"digestset",
"reference",
"registry/api/errcode",
]
pruneopts = "UT"
revision = "0d3efadf0154c2b8a4e7b6621fff9809655cc580"
[[projects]]
digest = "1:10f9c98f627e9697ec23b7973a683324f1d901dd9bace4a71405c0b2ec554303"
name = "github.com/docker/docker"
packages = [
"pkg/homedir",
"pkg/idtools",
"pkg/mount",
"pkg/system",
]
pruneopts = "UT"
revision = "ea84732a77251e0d7af278e2b7df1d6a59fca46b"
version = "v19.03.5"
[[projects]]
digest = "1:9f3f49b4e32d3da2dd6ed07cc568627b53cc80205c0dcf69f4091f027416cb60"
name = "github.com/docker/docker-credential-helpers"
packages = [
"client",
"credentials",
]
pruneopts = "UT"
revision = "54f0238b6bf101fc3ad3b34114cb5520beb562f5"
version = "v0.6.3"
[[projects]]
digest = "1:ade935c55cd6d0367c843b109b09c9d748b1982952031414740750fdf94747eb"
name = "github.com/docker/go-connections"
packages = ["nat"]
pruneopts = "UT"
revision = "7395e3f8aa162843a74ed6d48e79627d9792ac55"
version = "v0.4.0"
[[projects]]
digest = "1:0938aba6e09d72d48db029d44dcfa304851f52e2d67cda920436794248e92793"
name = "github.com/docker/go-events"
packages = ["."]
pruneopts = "UT"
revision = "9461782956ad83b30282bf90e31fa6a70c255ba9"
[[projects]]
digest = "1:e95ef557dc3120984bb66b385ae01b4bb8ff56bcde28e7b0d1beed0cccc4d69f"
name = "github.com/docker/go-units"
packages = ["."]
pruneopts = "UT"
revision = "519db1ee28dcc9fd2474ae59fca29a810482bfb1"
version = "v0.4.0"
[[projects]]
digest = "1:fa6faf4a2977dc7643de38ae599a95424d82f8ffc184045510737010a82c4ecd"
name = "github.com/gogo/googleapis"
packages = ["google/rpc"]
pruneopts = "UT"
revision = "d31c731455cb061f42baff3bda55bad0118b126b"
version = "v1.2.0"
[[projects]]
digest = "1:4107f4e81e8fd2e80386b4ed56b05e3a1fe26ecc7275fe80bb9c3a80a7344ff4"
name = "github.com/gogo/protobuf"
packages = [
"proto",
"sortkeys",
"types",
]
pruneopts = "UT"
revision = "ba06b47c162d49f2af050fb4c75bcbc86a159d5c"
version = "v1.2.1"
[[projects]]
branch = "master"
digest = "1:b7cb6054d3dff43b38ad2e92492f220f57ae6087ee797dca298139776749ace8"
name = "github.com/golang/groupcache"
packages = ["lru"]
pruneopts = "UT"
revision = "611e8accdfc92c4187d399e95ce826046d4c8d73"
[[projects]]
digest = "1:f5ce1529abc1204444ec73779f44f94e2fa8fcdb7aca3c355b0c95947e4005c6"
name = "github.com/golang/protobuf"
packages = [
"proto",
"ptypes",
"ptypes/any",
"ptypes/duration",
"ptypes/timestamp",
]
pruneopts = "UT"
revision = "6c65a5562fc06764971b7c5d05c76c75e84bdbf7"
version = "v1.3.2"
[[projects]]
digest = "1:cbec35fe4d5a4fba369a656a8cd65e244ea2c743007d8f6c1ccb132acf9d1296"
name = "github.com/gorilla/mux"
packages = ["."]
pruneopts = "UT"
revision = "00bdffe0f3c77e27d2cf6f5c70232a2d3e4d9c15"
version = "v1.7.3"
[[projects]]
digest = "1:1a7059d684f8972987e4b6f0703083f207d63f63da0ea19610ef2e6bb73db059"
name = "github.com/imdario/mergo"
packages = ["."]
pruneopts = "UT"
revision = "66f88b4ae75f5edcc556623b96ff32c06360fbb7"
version = "v0.3.9"
[[projects]]
digest = "1:870d441fe217b8e689d7949fef6e43efbc787e50f200cb1e70dbca9204a1d6be"
name = "github.com/inconshreveable/mousetrap"
packages = ["."]
pruneopts = "UT"
revision = "76626ae9c91c4f2a10f34cad8ce83ea42c93bb75"
version = "v1.0"
[[projects]]
digest = "1:31e761d97c76151dde79e9d28964a812c46efc5baee4085b86f68f0c654450de"
name = "github.com/konsorten/go-windows-terminal-sequences"
packages = ["."]
pruneopts = "UT"
revision = "f55edac94c9bbba5d6182a4be46d86a2c9b5b50e"
version = "v1.0.2"
[[projects]]
digest = "1:528e84b49342ec33c96022f8d7dd4c8bd36881798afbb44e2744bda0ec72299c"
name = "github.com/mattn/go-shellwords"
packages = ["."]
pruneopts = "UT"
revision = "28e4fdf351f0744b1249317edb45e4c2aa7a5e43"
version = "v1.0.10"
[[projects]]
digest = "1:dd34285411cd7599f1fe588ef9451d5237095963ecc85c1212016c6769866306"
name = "github.com/mitchellh/mapstructure"
packages = ["."]
pruneopts = "UT"
revision = "20e21c67c4d0e1b4244f83449b7cdd10435ee998"
version = "v1.3.1"
[[projects]]
digest = "1:906eb1ca3c8455e447b99a45237b2b9615b665608fd07ad12cce847dd9a1ec43"
name = "github.com/morikuni/aec"
packages = ["."]
pruneopts = "UT"
revision = "39771216ff4c63d11f5e604076f9c45e8be1067b"
version = "v1.0.0"
[[projects]]
digest = "1:bc62c2c038cc8ae51b68f6d52570501a763bb71e78736a9f65d60762429864a9"
name = "github.com/opencontainers/go-digest"
packages = ["."]
pruneopts = "UT"
revision = "c9281466c8b2f606084ac71339773efd177436e7"
[[projects]]
digest = "1:70711188c19c53147099d106169d6a81941ed5c2658651432de564a7d60fd288"
name = "github.com/opencontainers/image-spec"
packages = [
"identity",
"specs-go",
"specs-go/v1",
]
pruneopts = "UT"
revision = "d60099175f88c47cd379c4738d158884749ed235"
version = "v1.0.1"
[[projects]]
digest = "1:18d6ebfbabffccba7318a4e26028b0d41f23ff359df3dc07a53b37a9f3a4a994"
name = "github.com/opencontainers/runc"
packages = [
"libcontainer/system",
"libcontainer/user",
]
pruneopts = "UT"
revision = "d736ef14f0288d6993a1845745d6756cfc9ddd5a"
version = "v1.0.0-rc9"
[[projects]]
digest = "1:7a58202c5cdf3d2c1eb0621fe369315561cea7f036ad10f0f0479ac36bcc95eb"
name = "github.com/opencontainers/runtime-spec"
packages = ["specs-go"]
pruneopts = "UT"
revision = "29686dbc5559d93fb1ef402eeda3e35c38d75af4"
[[projects]]
digest = "1:cdf3df431e70077f94e14a99305808e3d13e96262b4686154970f448f7248842"
name = "github.com/openfaas/faas"
packages = ["gateway/requests"]
pruneopts = "UT"
revision = "80b6976c106370a7081b2f8e9099a6ea9638e1f3"
version = "0.18.10"
[[projects]]
digest = "1:4d972c6728f8cbaded7d2ee6349fbe5f9278cabcd51d1ecad97b2e79c72bea9d"
name = "github.com/openfaas/faas-provider"
packages = [
".",
"auth",
"httputil",
"logs",
"proxy",
"types",
]
pruneopts = "UT"
revision = "db19209aa27f42a9cf6a23448fc2b8c9cc4fbb5d"
version = "v0.15.1"
[[projects]]
digest = "1:cf31692c14422fa27c83a05292eb5cbe0fb2775972e8f1f8446a71549bd8980b"
name = "github.com/pkg/errors"
packages = ["."]
pruneopts = "UT"
revision = "ba968bfe8b2f7e042a574c888954fccecfa385b4"
version = "v0.8.1"
[[projects]]
digest = "1:044c51736e2688a3e4f28f72537f8a7b3f9c188fab4477d5334d92dfe2c07ed5"
name = "github.com/sethvargo/go-password"
packages = ["password"]
pruneopts = "UT"
revision = "07c3d521e892540e71469bb0312866130714c038"
version = "v0.1.3"
[[projects]]
digest = "1:fd61cf4ae1953d55df708acb6b91492d538f49c305b364a014049914495db426"
name = "github.com/sirupsen/logrus"
packages = ["."]
pruneopts = "UT"
revision = "8bdbc7bcc01dcbb8ec23dc8a28e332258d25251f"
version = "v1.4.1"
[[projects]]
digest = "1:e096613fb7cf34743d49af87d197663cfccd61876e2219853005a57baedfa562"
name = "github.com/spf13/cobra"
packages = ["."]
pruneopts = "UT"
revision = "f2b07da1e2c38d5f12845a4f607e2e1018cbb1f5"
version = "v0.0.5"
[[projects]]
digest = "1:524b71991fc7d9246cc7dc2d9e0886ccb97648091c63e30eef619e6862c955dd"
name = "github.com/spf13/pflag"
packages = ["."]
pruneopts = "UT"
revision = "2e9d26c8c37aae03e3f9d4e90b7116f5accb7cab"
version = "v1.0.5"
[[projects]]
branch = "master"
digest = "1:e14e467ed00ab98665623c5060fa17e3d7079be560ffc33cabafd05d35894f05"
name = "github.com/syndtr/gocapability"
packages = ["capability"]
pruneopts = "UT"
revision = "d98352740cb2c55f81556b63d4a1ec64c5a319c2"
[[projects]]
digest = "1:1314b5ef1c0b25257ea02e454291bf042478a48407cfe3ffea7e20323bbf5fdf"
name = "github.com/vishvananda/netlink"
packages = [
".",
"nl",
]
pruneopts = "UT"
revision = "f049be6f391489d3f374498fe0c8df8449258372"
version = "v1.1.0"
[[projects]]
branch = "master"
digest = "1:975cb0c04431bf92e60b636a15897c4a3faba9f7dc04da505646630ac91d29d3"
name = "github.com/vishvananda/netns"
packages = ["."]
pruneopts = "UT"
revision = "0a2b9b5464df8343199164a0321edf3313202f7e"
[[projects]]
branch = "master"
digest = "1:87fe9bca786484cef53d52adeec7d1c52bc2bfbee75734eddeb75fc5c7023871"
name = "github.com/xeipuuv/gojsonpointer"
packages = ["."]
pruneopts = "UT"
revision = "02993c407bfbf5f6dae44c4f4b1cf6a39b5fc5bb"
[[projects]]
branch = "master"
digest = "1:dc6a6c28ca45d38cfce9f7cb61681ee38c5b99ec1425339bfc1e1a7ba769c807"
name = "github.com/xeipuuv/gojsonreference"
packages = ["."]
pruneopts = "UT"
revision = "bd5ef7bd5415a7ac448318e64f11a24cd21e594b"
[[projects]]
digest = "1:a8a0ed98532819a3b0dc5cf3264a14e30aba5284b793ba2850d6f381ada5f987"
name = "github.com/xeipuuv/gojsonschema"
packages = ["."]
pruneopts = "UT"
revision = "82fcdeb203eb6ab2a67d0a623d9c19e5e5a64927"
version = "v1.2.0"
[[projects]]
digest = "1:aed53a5fa03c1270457e331cf8b7e210e3088a2278fec552c5c5d29c1664e161"
name = "go.opencensus.io"
packages = [
".",
"internal",
"trace",
"trace/internal",
"trace/tracestate",
]
pruneopts = "UT"
revision = "aad2c527c5defcf89b5afab7f37274304195a6b2"
version = "v0.22.2"
[[projects]]
branch = "master"
digest = "1:676f320d34ccfa88bfa6d04bdf388ed7062af175355c805ef57ccda1a3f13432"
name = "golang.org/x/net"
packages = [
"context",
"context/ctxhttp",
"http/httpguts",
"http2",
"http2/hpack",
"idna",
"internal/timeseries",
"trace",
]
pruneopts = "UT"
revision = "c0dbc17a35534bf2e581d7a942408dc936316da4"
[[projects]]
digest = "1:d6b0cfc5ae30841c4b116ac589629f56f8add0955a39f11d8c0d06ca67f5b3d5"
name = "golang.org/x/sync"
packages = [
"errgroup",
"semaphore",
]
pruneopts = "UT"
revision = "42b317875d0fa942474b76e1b46a6060d720ae6e"
[[projects]]
branch = "master"
digest = "1:a76bac71eb452a046b47f82336ba792d8de988688a912f3fd0e8ec8e57fe1bb4"
name = "golang.org/x/sys"
packages = [
"unix",
"windows",
]
pruneopts = "UT"
revision = "af0d71d358abe0ba3594483a5d519f429dbae3e9"
[[projects]]
digest = "1:8d8faad6b12a3a4c819a3f9618cb6ee1fa1cfc33253abeeea8b55336721e3405"
name = "golang.org/x/text"
packages = [
"collate",
"collate/build",
"internal/colltab",
"internal/gen",
"internal/language",
"internal/language/compact",
"internal/tag",
"internal/triegen",
"internal/ucd",
"language",
"secure/bidirule",
"transform",
"unicode/bidi",
"unicode/cldr",
"unicode/norm",
"unicode/rangetable",
]
pruneopts = "UT"
revision = "342b2e1fbaa52c93f31447ad2c6abc048c63e475"
version = "v0.3.2"
[[projects]]
branch = "master"
digest = "1:583a0c80f5e3a9343d33aea4aead1e1afcc0043db66fdf961ddd1fe8cd3a4faf"
name = "google.golang.org/genproto"
packages = ["googleapis/rpc/status"]
pruneopts = "UT"
revision = "b31c10ee225f87dbb9f5f878ead9d64f34f5cbbb"
[[projects]]
digest = "1:48eafc052e46b4ebbc7882553873cf6198203e528627cefc94dcaf8553addd19"
name = "google.golang.org/grpc"
packages = [
".",
"balancer",
"balancer/base",
"balancer/roundrobin",
"binarylog/grpc_binarylog_v1",
"codes",
"connectivity",
"credentials",
"credentials/internal",
"encoding",
"encoding/proto",
"grpclog",
"health/grpc_health_v1",
"internal",
"internal/backoff",
"internal/balancerload",
"internal/binarylog",
"internal/channelz",
"internal/envconfig",
"internal/grpcrand",
"internal/grpcsync",
"internal/syscall",
"internal/transport",
"keepalive",
"metadata",
"naming",
"peer",
"resolver",
"resolver/dns",
"resolver/passthrough",
"serviceconfig",
"stats",
"status",
"tap",
]
pruneopts = "UT"
revision = "6eaf6f47437a6b4e2153a190160ef39a92c7eceb"
version = "v1.23.0"
[[projects]]
digest = "1:d7f1bd887dc650737a421b872ca883059580e9f8314d601f88025df4f4802dce"
name = "gopkg.in/yaml.v2"
packages = ["."]
pruneopts = "UT"
revision = "0b1645d91e851e735d3e23330303ce81f70adbe3"
version = "v2.3.0"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
input-imports = [
"github.com/alexellis/go-execute/pkg/v1",
"github.com/alexellis/k3sup/pkg/env",
"github.com/compose-spec/compose-go/loader",
"github.com/compose-spec/compose-go/types",
"github.com/containerd/containerd",
"github.com/containerd/containerd/cio",
"github.com/containerd/containerd/containers",
"github.com/containerd/containerd/errdefs",
"github.com/containerd/containerd/namespaces",
"github.com/containerd/containerd/oci",
"github.com/containerd/containerd/remotes",
"github.com/containerd/containerd/remotes/docker",
"github.com/containerd/containerd/runtime/v2/logging",
"github.com/containerd/go-cni",
"github.com/coreos/go-systemd/journal",
"github.com/docker/cli/cli/config",
"github.com/docker/cli/cli/config/configfile",
"github.com/docker/distribution/reference",
"github.com/gorilla/mux",
"github.com/morikuni/aec",
"github.com/opencontainers/runtime-spec/specs-go",
"github.com/openfaas/faas-provider",
"github.com/openfaas/faas-provider/logs",
"github.com/openfaas/faas-provider/proxy",
"github.com/openfaas/faas-provider/types",
"github.com/openfaas/faas/gateway/requests",
"github.com/pkg/errors",
"github.com/sethvargo/go-password/password",
"github.com/spf13/cobra",
"github.com/spf13/pflag",
"github.com/vishvananda/netlink",
"github.com/vishvananda/netns",
"golang.org/x/sys/unix",
]
solver-name = "gps-cdcl"
solver-version = 1

47
Gopkg.toml Normal file
View File

@ -0,0 +1,47 @@
[prune]
go-tests = true
unused-packages = true
[[constraint]]
name = "github.com/containerd/containerd"
version = "1.3.2"
[[constraint]]
name = "github.com/morikuni/aec"
version = "1.0.0"
[[constraint]]
name = "github.com/spf13/cobra"
version = "0.0.5"
[[constraint]]
name = "github.com/alexellis/k3sup"
version = "0.9.3"
[[constraint]]
name = "github.com/alexellis/go-execute"
version = "0.4.0"
[[constraint]]
name = "github.com/gorilla/mux"
version = "1.7.3"
[[constraint]]
name = "github.com/openfaas/faas"
version = "0.18.7"
[[constraint]]
name = "github.com/sethvargo/go-password"
version = "0.1.3"
[[constraint]]
branch = "master"
name = "github.com/containerd/go-cni"
[[constraint]]
name = "github.com/openfaas/faas-provider"
version = "v0.15.1"
[[constraint]]
name = "github.com/docker/cli"
version = "19.3.5"

View File

@ -1,10 +1,8 @@
License for faasd contributions from OpenFaaS Ltd - 2017, 2029-2024, see: EULA.md
Only third-party contributions to source code are licensed MIT:
MIT License
Copyright (c) 2020 Alex Ellis
Copyright (c) 2020 OpenFaaS Ltd
Copyright (c) 2020 OpenFaas Author(s)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

View File

@ -1,48 +1,36 @@
Version := $(shell git describe --tags --dirty)
GitCommit := $(shell git rev-parse HEAD)
LDFLAGS := "-s -w -X github.com/openfaas/faasd/pkg.Version=$(Version) -X github.com/openfaas/faasd/pkg.GitCommit=$(GitCommit)"
CONTAINERD_VER := 1.7.22
CNI_VERSION := v0.9.1
LDFLAGS := "-s -w -X main.Version=$(Version) -X main.GitCommit=$(GitCommit)"
CONTAINERD_VER := 1.3.4
CNI_VERSION := v0.8.6
ARCH := amd64
export GO111MODULE=on
.PHONY: all
all: test dist hashgen
.PHONY: publish
publish: dist hashgen
all: local
local:
CGO_ENABLED=0 GOOS=linux go build -mod=vendor -o bin/faasd
CGO_ENABLED=0 GOOS=linux go build -o bin/faasd
.PHONY: test
test:
CGO_ENABLED=0 GOOS=linux go test -mod=vendor -ldflags $(LDFLAGS) ./...
.PHONY: dist-local
dist-local:
CGO_ENABLED=0 GOOS=linux go build -mod=vendor -ldflags $(LDFLAGS) -o bin/faasd
CGO_ENABLED=0 GOOS=linux go test -ldflags $(LDFLAGS) ./...
.PHONY: dist
dist:
CGO_ENABLED=0 GOOS=linux go build -mod=vendor -ldflags $(LDFLAGS) -o bin/faasd
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -mod=vendor -ldflags $(LDFLAGS) -o bin/faasd-arm64
.PHONY: hashgen
hashgen:
for f in bin/faasd*; do shasum -a 256 $$f > $$f.sha256; done
CGO_ENABLED=0 GOOS=linux go build -ldflags $(LDFLAGS) -a -installsuffix cgo -o bin/faasd
CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 go build -ldflags $(LDFLAGS) -a -installsuffix cgo -o bin/faasd-armhf
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -ldflags $(LDFLAGS) -a -installsuffix cgo -o bin/faasd-arm64
.PHONY: prepare-test
prepare-test:
curl -sLSf https://github.com/containerd/containerd/releases/download/v$(CONTAINERD_VER)/containerd-$(CONTAINERD_VER)-linux-amd64.tar.gz > /tmp/containerd.tar.gz && sudo tar -xvf /tmp/containerd.tar.gz -C /usr/local/bin/ --strip-components=1
curl -SLfs https://raw.githubusercontent.com/containerd/containerd/v1.7.0/containerd.service | sudo tee /etc/systemd/system/containerd.service
curl -sLSf https://github.com/containerd/containerd/releases/download/v$(CONTAINERD_VER)/containerd-$(CONTAINERD_VER).linux-amd64.tar.gz > /tmp/containerd.tar.gz && sudo tar -xvf /tmp/containerd.tar.gz -C /usr/local/bin/ --strip-components=1
curl -SLfs https://raw.githubusercontent.com/containerd/containerd/v1.3.2/containerd.service | sudo tee /etc/systemd/system/containerd.service
sudo systemctl daemon-reload && sudo systemctl start containerd
sudo /sbin/sysctl -w net.ipv4.conf.all.forwarding=1
sudo mkdir -p /opt/cni/bin
curl -sSL https://github.com/containernetworking/plugins/releases/download/$(CNI_VERSION)/cni-plugins-linux-$(ARCH)-$(CNI_VERSION).tgz | sudo tar -xz -C /opt/cni/bin
sudo cp bin/faasd /usr/local/bin/
sudo /usr/local/bin/faasd install
sudo cp $(GOPATH)/src/github.com/openfaas/faasd/bin/faasd /usr/local/bin/
cd $(GOPATH)/src/github.com/openfaas/faasd/ && sudo /usr/local/bin/faasd install
sudo systemctl status -l containerd --no-pager
sudo journalctl -u faasd-provider --no-pager
sudo systemctl status -l faasd-provider --no-pager
@ -64,16 +52,5 @@ test-e2e:
/usr/local/bin/faas-cli remove figlet
sleep 3
/usr/local/bin/faas-cli list
sleep 3
journalctl -t openfaas-fn:figlet --no-pager
# Removed due to timing issue in CI on GitHub Actions
# /usr/local/bin/faas-cli logs figlet --since 15m --follow=false | grep Forking
verify-compose:
@echo Verifying docker-compose.yaml images in remote registries && \
arkade chart verify --verbose=$(VERBOSE) -f ./docker-compose.yaml
upgrade-compose:
@echo Checking for newer images in remote registries && \
arkade chart upgrade -f ./docker-compose.yaml --write
sleep 1
/usr/local/bin/faas-cli logs figlet --follow=false | grep Forking

308
README.md
View File

@ -1,71 +1,101 @@
# faasd - a lightweight & portable faas engine
# faasd - Serverless for everyone else
[![Sponsor faasd](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&link=https://github.com/sponsors/openfaas)](https://github.com/sponsors/openfaas)
[![Build Status](https://github.com/openfaas/faasd/workflows/build/badge.svg?branch=master)](https://github.com/openfaas/faasd/actions)
faasd is built for everyone else, for those who have no desire to manage expensive infrastructure.
[![Build Status](https://travis-ci.com/openfaas/faasd.svg?branch=master)](https://travis-ci.com/openfaas/faasd)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![OpenFaaS](https://img.shields.io/badge/openfaas-serverless-blue.svg)](https://www.openfaas.com)
![Downloads](https://img.shields.io/github/downloads/openfaas/faasd/total)
faasd is [OpenFaaS](https://github.com/openfaas/) reimagined, but without the cost and complexity of Kubernetes. It runs on a single host with very modest requirements, making it fast and easy to manage. Under the hood it uses [containerd](https://containerd.io/) and [Container Networking Interface (CNI)](https://github.com/containernetworking/cni) along with the same core OpenFaaS components from the main project.
faasd is [OpenFaaS](https://github.com/openfaas/) reimagined, but without the cost and complexity of Kubernetes. It runs on a single host with very modest requirements, making it very fast and easy to manage. Under the hood it uses [containerd](https://containerd.io/) and [Container Networking Interface (CNI)](https://github.com/containernetworking/cni) along with the same core OpenFaaS components from the main project.
![faasd logo](docs/media/social.png)
## When should you use faasd over OpenFaaS on Kubernetes?
## Use-cases and tutorials
* You have a cost sensitive project - run faasd on a 5-10 USD VPS or on your Raspberry Pi
* When you just need a few functions or microservices, without the cost of a cluster
* When you don't have the bandwidth to learn or manage Kubernetes
* To deploy embedded apps in IoT and edge use-cases
* To shrink-wrap applications for use with a customer or client
faasd is just another way to run OpenFaaS, so many things you read in the docs or in blog posts will work the same way.
faasd does not create the same maintenance burden you'll find with maintaining, upgrading, and securing a Kubernetes cluster. You can deploy it and walk away, in the worst case, just deploy a new VM and deploy your functions again.
Videos and overviews:
## About faasd
* [Exploring of serverless use-cases from commercial and personal users (YouTube)](https://www.youtube.com/watch?v=mzuXVuccaqI)
* [Meet faasd. Look Ma No Kubernetes! (YouTube)](https://www.youtube.com/watch?v=ZnZJXI377ak)
Use-cases and tutorials:
* [Serverless Node.js that you can run anywhere](https://www.openfaas.com/blog/serverless-nodejs/)
* [Simple Serverless with Golang Functions and Microservices](https://www.openfaas.com/blog/golang-serverless/)
* [Build a Flask microservice with OpenFaaS](https://www.openfaas.com/blog/openfaas-flask/)
* [Get started with Java 11 and Vert.x on Kubernetes with OpenFaaS](https://www.openfaas.com/blog/get-started-with-java-openjdk11/)
* [Deploy to faasd via GitHub Actions](https://www.openfaas.com/blog/openfaas-functions-with-github-actions/)
* [Scrape and automate websites with Puppeteer](https://www.openfaas.com/blog/puppeteer-scraping/)
Additional resources:
* The official handbook - [Serverless For Everyone Else](https://openfaas.gumroad.com/l/serverless-for-everyone-else)
* For reference: [OpenFaaS docs](https://docs.openfaas.com)
* For use-cases and tutorials: [OpenFaaS blog](https://openfaas.com/blog/)
* For self-paced learning: [OpenFaaS workshop](https://github.com/openfaas/workshop/)
### About faasd
* faasd is a single static Golang binary, which runs on Linux with systemd
* is a single Golang binary
* uses the same core components and ecosystem of OpenFaaS
* uses containerd for its runtime and CNI for networking
* is multi-arch, so works on Intel `x86_64` and Arm out the box
* runs stateful containers through its `docker-compose.yaml` file like Grafana, MongoDB, InfluxDB, or Postgres, etc
* is multi-arch, so works on Intel `x86_64` and ARM out the box
* can be set-up and left alone to run your applications
Most importantly, it's easy to manage so you can set it up and leave it alone to run your functions.
![demo](https://pbs.twimg.com/media/EPNQz00W4AEwDxM?format=jpg&name=small)
[![demo](https://pbs.twimg.com/media/EPNQz00W4AEwDxM?format=jpg&name=medium)](https://www.youtube.com/watch?v=WX1tZoSXy8E)
> Demo of faasd running in KVM
> Demo of faasd running asynchronous functions
## Tutorials
Watch the video: [faasd walk-through with cloud-init and Multipass](https://www.youtube.com/watch?v=WX1tZoSXy8E)
### Get started on DigitalOcean, or any other IaaS
### What does faasd deploy?
If your IaaS supports `user_data` aka "cloud-init", then this guide is for you. If not, then checkout the approach and feel free to run each step manually.
* [Build a Serverless appliance with faasd](https://blog.alexellis.io/deploy-serverless-faasd-with-cloud-init/)
### Run locally on MacOS, Linux, or Windows with multipass
* [Get up and running with your own faasd installation on your Mac/Ubuntu or Windows with cloud-config](/docs/MULTIPASS.md)
### Get started on armhf / Raspberry Pi
You can run this tutorial on your Raspberry Pi, or adapt the steps for a regular Linux VM/VPS host.
* [faasd - lightweight Serverless for your Raspberry Pi](https://blog.alexellis.io/faasd-for-lightweight-serverless/)
### Terraform for DigitalOcean
Automate everything within < 60 seconds and get a public URL and IP address back. Customise as required, or adapt to your preferred cloud such as AWS EC2.
* [Provision faasd 0.8.1 on DigitalOcean with Terraform 0.12.0](docs/bootstrap/README.md)
* [Provision faasd on DigitalOcean with built-in TLS support](docs/bootstrap/digitalocean-terraform/README.md)
### A note on private repos / registries
To use private image repos, `~/.docker/config.json` needs to be copied to `/var/lib/faasd/.docker/config.json`.
If you'd like to set up your own private registry, [see this tutorial](https://blog.alexellis.io/get-a-tls-enabled-docker-registry-in-5-minutes/).
Beware that running `docker login` on MacOS and Windows may create an empty file with your credentials stored in the system helper.
Alternatively, use you can use the `registry-login` command from the OpenFaaS Cloud bootstrap tool (ofc-bootstrap):
```bash
curl -sLSf https://raw.githubusercontent.com/openfaas-incubator/ofc-bootstrap/master/get.sh | sudo sh
ofc-bootstrap registry-login --username <your-registry-username> --password-stdin
# (the enter your password and hit return)
```
The file will be created in `./credentials/`
### Logs for functions
You can view the logs of functions using `journalctl`:
```bash
journalctl -t openfaas-fn:FUNCTION_NAME
faas-cli store deploy figlet
journalctl -t openfaas-fn:figlet -f &
echo logs | faas-cli invoke figlet
```
## What does faasd deploy?
* faasd - itself, and its [faas-provider](https://github.com/openfaas/faas-provider) for containerd - CRUD for functions and services, implements the OpenFaaS REST API
* [Prometheus](https://github.com/prometheus/prometheus) - for monitoring of services, metrics, scaling and dashboards
* [OpenFaaS CE Gateway](https://github.com/openfaas/faas/tree/master/gateway) - the UI portal, CLI, and other OpenFaaS tooling can talk to this.
* [OpenFaaS CE queue-worker for NATS Streaming](https://github.com/openfaas/nats-queue-worker) - run your invocations in the background without adding any code. See also: [asynchronous invocations](https://docs.openfaas.com/reference/triggers/#async-nats-streaming)
* [OpenFaaS Gateway](https://github.com/openfaas/faas/tree/master/gateway) - the UI portal, CLI, and other OpenFaaS tooling can talk to this.
* [OpenFaaS queue-worker for NATS](https://github.com/openfaas/nats-queue-worker) - run your invocations in the background without adding any code. See also: [asynchronous invocations](https://docs.openfaas.com/reference/triggers/#async-nats-streaming)
* [NATS](https://nats.io) for asynchronous processing and queues
## Licensing & EULA for OpenFaaS (faasd)
faasd (which is a distribution of OpenFaaS) is licensed under the [OpenFaaS Community Edition (CE) EULA](https://github.com/openfaas/faas/blob/master/EULA.md), [additional clauses apply](EULA.md) to enable faasd to be used for Personal Use and within Small Business Environments.
A separate OpenFaaS Standard license is available for commercial use-cases, which is required for certain commercial use.
### When should you use faasd over OpenFaaS on Kubernetes?
faasd relies on industry-standard tools for running containers:
You'll also need:
* [CNI](https://github.com/containernetworking/plugins)
* [containerd](https://github.com/containerd/containerd)
@ -73,119 +103,15 @@ faasd relies on industry-standard tools for running containers:
You can use the standard [faas-cli](https://github.com/openfaas/faas-cli) along with pre-packaged functions from *the Function Store*, or build your own using any OpenFaaS template.
* To deploy microservices and functions that you can update and monitor remotely
* When you don't have the bandwidth to learn or manage Kubernetes
* To deploy embedded apps in IoT and edge use-cases
* To distribute applications to a customer or client
* You have a cost sensitive project - run faasd on a 1GB VM for 5-10 USD / mo or on your Raspberry Pi
* When you just need a few functions or microservices, without the cost of a cluster
### Manual / developer instructions
faasd does not create the same maintenance burden you'll find with maintaining, upgrading, and securing a Kubernetes cluster. You can deploy it and walk away, in the worst case, just deploy a new VM and deploy your functions again.
See [here for manual / developer instructions](docs/DEV.md)
You can learn more about supported OpenFaaS features in the [ROADMAP.md](/docs/ROADMAP.md)
## Getting help
## Learning faasd
### Docs
You can find various resources to learn about faasd for free, however the official handbook is the most comprehensive guide to getting started with faasd and OpenFaaS.
["Serverless For Everyone Else"](https://openfaas.gumroad.com/l/serverless-for-everyone-else) is the official handbook and was written to contribute funds towards the upkeep and maintenance of the project.
### The official handbook and docs for faasd
<a href="https://openfaas.gumroad.com/l/serverless-for-everyone-else">
<img src="https://www.alexellis.io/serverless.png" width="40%"></a>
You'll learn how to deploy code in any language, lift and shift Dockerfiles, run requests in queues, write background jobs and to integrate with databases. faasd packages the same code as OpenFaaS, so you get built-in metrics for your HTTP endpoints, a user-friendly CLI, pre-packaged functions and templates from the store and a UI.
Topics include:
* Should you deploy to a VPS or Raspberry Pi?
* Deploying your server with bash, cloud-init or terraform
* Using a private container registry
* Finding functions in the store
* Building your first function with Node.js
* Using environment variables for configuration
* Using secrets from functions, and enabling authentication tokens
* Customising templates
* Monitoring your functions with Grafana and Prometheus
* Scheduling invocations and background jobs
* Tuning timeouts, parallelism, running tasks in the background
* Adding TLS to faasd and custom domains for functions
* Self-hosting on your Raspberry Pi
* Adding a database for storage with InfluxDB and Postgresql
* Troubleshooting and logs
* CI/CD with GitHub Actions and multi-arch
* Taking things further, community and case-studies
View sample pages, reviews and testimonials on Gumroad:
["Serverless For Everyone Else"](https://openfaas.gumroad.com/l/serverless-for-everyone-else)
### Deploy OpenFaaS Edge (commercial distribution of faasd)
OpenFaaS Edge is a commercial distribution of faasd, with enhancements and additional features from OpenFaaS Pro. The [OpenFaaS Pro EULA applies](https://github.com/openfaas/faas/blob/master/pro/EULA.md).
* Upgraded Pro components from OpenFaaS Standard: Gateway, Cron Connector, JetStream Queue Worker and Classic Scale to Zero
* Deploy up to 250 functions per installation
* Configure private DNS servers
* Airgap-friendly with installation bundled in an OCI image
* Detailed RAM/CPU metrics for stateful containers, and functions, including Out Of Memory (OOM) events
* Multiple namespace support
This version is intended for resale as part of a wider solution, and to be deployed both into industrial and on-premises environments.
Individual [GitHub Sponsors of OpenFaaS](https://github.com/sponsors/openfaas) (25 USD / mo and higher) can use OpenFaaS Edge for personal use.
You can install OpenFaaS Edge with the following script:
```bash
curl -sLSf \
https://raw.githubusercontent.com/openfaas/faasd/refs/heads/master/hack/install-edge.sh \
-o install-edge.sh && \
chmod +x install-edge.sh
sudo -E ./install-edge.sh
```
*For an offline installation*
Copy the OCI bundle and the install-edge.sh script to the remote server.
Then mirror the various images from docker-compose.yaml into your private registry, and update the references from i.e. `image: ghcr.io/openfaasltd/gateway` to the equivalents in your registry.
If your system is unable to install apt, yum, or pacman packages, due to limited network access, then set the `SKIP_OS` environment to 1. The list of packages is available in the `install_required_packages` section of the script.
### Deploy faasd CE
faasd-ce supports 15 functions and needs a computer with a stable Internet connection to run. There are some restrictions on commercial use, but [individuals and certain small businesses can use it for free](EULA.md).
The easiest way to deploy faasd is with cloud-init, we give several examples below, and post IaaS platforms will accept "user-data" pasted into their UI, or via their API.
For trying it out on MacOS or Windows, we recommend using [multipass](https://multipass.run) to run faasd in a VM.
If you don't use cloud-init, or have already created your Linux server you can use the installation script as per below:
```bash
git clone https://github.com/openfaas/faasd --depth=1
cd faasd
./hack/install.sh
```
> This approach also works for Raspberry Pi when using the 64-bit OS or Ubuntu 22.04 or higher.
It's recommended that you do not install Docker on the same host as faasd, since 1) they may both use different versions of containerd and 2) docker's networking rules can disrupt faasd's networking. When using faasd - make your faasd server a faasd server, and build container image on your laptop or in a CI pipeline.
#### Deployment tutorials
* [Use multipass on Windows, MacOS or Linux](/docs/MULTIPASS.md)
* [Deploy to DigitalOcean with Terraform and TLS](https://www.openfaas.com/blog/faasd-tls-terraform/)
* [Deploy to any IaaS with cloud-init](https://blog.alexellis.io/deploy-serverless-faasd-with-cloud-init/)
* [Deploy faasd to your Raspberry Pi](https://blog.alexellis.io/faasd-for-lightweight-serverless/)
Terraform scripts:
* [Provision faasd on DigitalOcean with Terraform](docs/bootstrap/README.md)
* [Provision faasd with TLS on DigitalOcean with Terraform](docs/bootstrap/digitalocean-terraform/README.md)
The [OpenFaaS docs](https://docs.openfaas.com/) provide a wealth of information and are kept up to date with new features.
### Function and template store
@ -193,19 +119,69 @@ For community functions see `faas-cli store --help`
For templates built by the community see: `faas-cli template store list`, you can also use the `dockerfile` template if you just want to migrate an existing service without the benefits of using a template.
### Workshop
[The OpenFaaS workshop](https://github.com/openfaas/workshop/) is a set of 12 self-paced labs and provides a great starting point for learning the features of openfaas. Not all features will be available or usable with faasd.
### Community support
Commercial users and solo business owners should become OpenFaaS GitHub Sponsors to receive regular email updates on changes, tutorials and new features.
An active community of almost 3000 users awaits you on Slack. Over 250 of those users are also contributors and help maintain the code.
If you are learning faasd, or want to share your use-case, you can join the OpenFaaS Slack community.
* [Join Slack](https://slack.openfaas.io/)
* [Become an OpenFaaS GitHub Sponsor](https://github.com/sponsors/openfaas/)
* [Join the weekly Office Hours call](https://docs.openfaas.com/community/)
## Roadmap
### Backlog, features, design limitations and any known issues
### Supported operations
For open backlog items, shipped features, design limitations and any known issues, see [ROADMAP.md](docs/ROADMAP.md)
* `faas login`
* `faas up`
* `faas list`
* `faas describe`
* `faas deploy --update=true --replace=false`
* `faas invoke --async`
* `faas invoke`
* `faas rm`
* `faas store list/deploy/inspect`
* `faas version`
* `faas namespace`
* `faas secret`
* `faas logs`
Want to build a patch without setting up a complete development environment? See [docs/PATCHES.md](docs/PATCHES.md)
Scale from and to zero is also supported. On a Dell XPS with a small, pre-pulled image unpausing an existing task took 0.19s and starting a task for a killed function took 0.39s. There may be further optimizations to be gained.
Other operations are pending development in the provider such as:
* `faas auth` - supported for Basic Authentication, but OAuth2 & OIDC require a patch
### Backlog
* [ ] [Store and retrieve annotations in function spec](https://github.com/openfaas/faasd/pull/86) - in progress
* [ ] Offer live rolling-updates, with zero downtime - requires moving to IDs vs. names for function containers
* [ ] An installer for faasd and dependencies - runc, containerd
* [ ] Monitor and restart any of the core components at runtime if the container stops
* [ ] Provide ufw rules / example for blocking access to everything but a reverse proxy to the gateway container
* [ ] Provide [simple Caddyfile example](https://blog.alexellis.io/https-inlets-local-endpoints/) in the README showing how to expose the faasd proxy on port 80/443 with TLS
### Known-issues
* [ ] [containerd can't pull image from Github Docker Package Registry](https://github.com/containerd/containerd/issues/3291)
### Completed
* [x] Provide a cloud-init configuration for faasd bootstrap
* [x] Configure core services from a docker-compose.yaml file
* [x] Store and fetch logs from the journal
* [x] Add support for using container images in third-party public registries
* [x] Add support for using container images in private third-party registries
* [x] Provide a cloud-config.txt file for automated deployments of `faasd`
* [x] Inject / manage IPs between core components for service to service communication - i.e. so Prometheus can scrape the OpenFaaS gateway - done via `/etc/hosts` mount
* [x] Add queue-worker and NATS
* [x] Create faasd.service and faasd-provider.service
* [x] Self-install / create systemd service via `faasd install`
* [x] Restart containers upon restart of faasd
* [x] Clear / remove containers and tasks with SIGTERM / SIGINT
* [x] Determine armhf/arm64 containers to run for gateway
* [x] Configure `basic_auth` to protect the OpenFaaS gateway and faasd-provider HTTP API
* [x] Setup custom working directory for faasd `/var/lib/faasd/`
* [x] Use CNI to create network namespaces and adapters
Are you looking to hack on faasd? Follow the [developer instructions](docs/DEV.md) for a manual installation, or use the `hack/install.sh` script and pick up from there.

View File

@ -7,10 +7,19 @@ package_update: true
packages:
- runc
- git
runcmd:
- curl -sfL https://raw.githubusercontent.com/openfaas/faasd/master/hack/install.sh | bash -s -
- curl -sLSf https://github.com/containerd/containerd/releases/download/v1.3.5/containerd-1.3.5-linux-amd64.tar.gz > /tmp/containerd.tar.gz && tar -xvf /tmp/containerd.tar.gz -C /usr/local/bin/ --strip-components=1
- curl -SLfs https://raw.githubusercontent.com/containerd/containerd/v1.3.5/containerd.service | tee /etc/systemd/system/containerd.service
- systemctl daemon-reload && systemctl start containerd
- systemctl enable containerd
- /sbin/sysctl -w net.ipv4.conf.all.forwarding=1
- mkdir -p /opt/cni/bin
- curl -sSL https://github.com/containernetworking/plugins/releases/download/v0.8.5/cni-plugins-linux-amd64-v0.8.5.tgz | tar -xz -C /opt/cni/bin
- mkdir -p /go/src/github.com/openfaas/
- cd /go/src/github.com/openfaas/ && git clone https://github.com/openfaas/faasd && git checkout 0.9.2
- curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.9.2/faasd" --output "/usr/local/bin/faasd" && chmod a+x "/usr/local/bin/faasd"
- cd /go/src/github.com/openfaas/faasd/ && /usr/local/bin/faasd install
- systemctl status -l containerd --no-pager
- journalctl -u faasd-provider --no-pager
- systemctl status -l faasd-provider --no-pager

View File

@ -50,45 +50,51 @@ func runInstall(_ *cobra.Command, _ []string) error {
return err
}
if err := binExists("/usr/local/bin/", "faasd"); err != nil {
err := binExists("/usr/local/bin/", "faasd")
if err != nil {
return err
}
if err := systemd.InstallUnit("faasd-provider", map[string]string{
err = systemd.InstallUnit("faasd-provider", map[string]string{
"Cwd": faasdProviderWd,
"SecretMountPath": path.Join(faasdwd, "secrets")}); err != nil {
"SecretMountPath": path.Join(faasdwd, "secrets")})
if err != nil {
return err
}
if err := systemd.InstallUnit("faasd", map[string]string{"Cwd": faasdwd}); err != nil {
err = systemd.InstallUnit("faasd", map[string]string{"Cwd": faasdwd})
if err != nil {
return err
}
if err := systemd.DaemonReload(); err != nil {
err = systemd.DaemonReload()
if err != nil {
return err
}
if err := systemd.Enable("faasd-provider"); err != nil {
err = systemd.Enable("faasd-provider")
if err != nil {
return err
}
if err := systemd.Enable("faasd"); err != nil {
err = systemd.Enable("faasd")
if err != nil {
return err
}
if err := systemd.Start("faasd-provider"); err != nil {
err = systemd.Start("faasd-provider")
if err != nil {
return err
}
if err := systemd.Start("faasd"); err != nil {
err = systemd.Start("faasd")
if err != nil {
return err
}
fmt.Println(`Check status with:
sudo journalctl -u faasd --lines 100 -f
Login with:
sudo -E cat /var/lib/faasd/secrets/basic-auth-password | faas-cli login -s`)
fmt.Println(`Login with:
sudo cat /var/lib/faasd/secrets/basic-auth-password | faas-cli login -s`)
return nil
}
@ -100,16 +106,7 @@ func binExists(folder, name string) error {
}
return nil
}
func ensureSecretsDir(folder string) error {
if _, err := os.Stat(folder); err != nil {
err = os.MkdirAll(folder, secretDirPermission)
if err != nil {
return err
}
}
return nil
}
func ensureWorkingDir(folder string) error {
if _, err := os.Stat(folder); err != nil {
err = os.MkdirAll(folder, workingDirectoryPermission)

View File

@ -1,8 +1,9 @@
package cmd
import (
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
@ -13,7 +14,6 @@ import (
"github.com/openfaas/faas-provider/logs"
"github.com/openfaas/faas-provider/proxy"
"github.com/openfaas/faas-provider/types"
faasd "github.com/openfaas/faasd/pkg"
"github.com/openfaas/faasd/pkg/cninetwork"
faasdlogs "github.com/openfaas/faasd/pkg/logs"
"github.com/openfaas/faasd/pkg/provider/config"
@ -21,152 +21,96 @@ import (
"github.com/spf13/cobra"
)
const secretDirPermission = 0755
func makeProviderCmd() *cobra.Command {
var command = &cobra.Command{
Use: "provider",
Short: "Run the faasd-provider",
}
command.RunE = runProviderE
command.PreRunE = preRunE
command.Flags().String("pull-policy", "Always", `Set to "Always" to force a pull of images upon deployment, or "IfNotPresent" to try to use a cached image.`)
command.RunE = func(_ *cobra.Command, _ []string) error {
pullPolicy, flagErr := command.Flags().GetString("pull-policy")
if flagErr != nil {
return flagErr
}
alwaysPull := false
if pullPolicy == "Always" {
alwaysPull = true
}
config, providerConfig, err := config.ReadFromEnv(types.OsEnv{})
if err != nil {
return err
}
log.Printf("faasd-provider starting..\tService Timeout: %s\n", config.WriteTimeout.String())
printVersion()
wd, err := os.Getwd()
if err != nil {
return err
}
writeHostsErr := ioutil.WriteFile(path.Join(wd, "hosts"),
[]byte(`127.0.0.1 localhost`), workingDirectoryPermission)
if writeHostsErr != nil {
return fmt.Errorf("cannot write hosts file: %s", writeHostsErr)
}
writeResolvErr := ioutil.WriteFile(path.Join(wd, "resolv.conf"),
[]byte(`nameserver 8.8.8.8`), workingDirectoryPermission)
if writeResolvErr != nil {
return fmt.Errorf("cannot write resolv.conf file: %s", writeResolvErr)
}
cni, err := cninetwork.InitNetwork()
if err != nil {
return err
}
client, err := containerd.New(providerConfig.Sock)
if err != nil {
return err
}
defer client.Close()
invokeResolver := handlers.NewInvokeResolver(client)
userSecretPath := path.Join(wd, "secrets")
bootstrapHandlers := types.FaaSHandlers{
FunctionProxy: proxy.NewHandlerFunc(*config, invokeResolver),
DeleteHandler: handlers.MakeDeleteHandler(client, cni),
DeployHandler: handlers.MakeDeployHandler(client, cni, userSecretPath, alwaysPull),
FunctionReader: handlers.MakeReadHandler(client),
ReplicaReader: handlers.MakeReplicaReaderHandler(client),
ReplicaUpdater: handlers.MakeReplicaUpdateHandler(client, cni),
UpdateHandler: handlers.MakeUpdateHandler(client, cni, userSecretPath, alwaysPull),
HealthHandler: func(w http.ResponseWriter, r *http.Request) {},
InfoHandler: handlers.MakeInfoHandler(Version, GitCommit),
ListNamespaceHandler: listNamespaces(),
SecretHandler: handlers.MakeSecretHandler(client, userSecretPath),
LogHandler: logs.NewLogHandlerFunc(faasdlogs.New(), config.ReadTimeout),
}
log.Printf("Listening on TCP port: %d\n", *config.TCPPort)
bootstrap.Serve(&bootstrapHandlers, config)
return nil
}
return command
}
func runProviderE(cmd *cobra.Command, _ []string) error {
config, providerConfig, err := config.ReadFromEnv(types.OsEnv{})
if err != nil {
return err
}
log.Printf("faasd-provider starting..\tService Timeout: %s\n", config.WriteTimeout.String())
printVersion()
wd, err := os.Getwd()
if err != nil {
return err
}
if err := os.WriteFile(path.Join(wd, "hosts"),
[]byte(`127.0.0.1 localhost`), workingDirectoryPermission); err != nil {
return fmt.Errorf("cannot write hosts file: %s", err)
}
if err := os.WriteFile(path.Join(wd, "resolv.conf"),
[]byte(`nameserver 8.8.8.8
nameserver 8.8.4.4`), workingDirectoryPermission); err != nil {
return fmt.Errorf("cannot write resolv.conf file: %s", err)
}
cni, err := cninetwork.InitNetwork()
if err != nil {
return err
}
client, err := containerd.New(providerConfig.Sock)
if err != nil {
return err
}
defer client.Close()
invokeResolver := handlers.NewInvokeResolver(client)
baseUserSecretsPath := path.Join(wd, "secrets")
if err := moveSecretsToDefaultNamespaceSecrets(
baseUserSecretsPath,
faasd.DefaultFunctionNamespace); err != nil {
return err
}
alwaysPull := true
bootstrapHandlers := types.FaaSHandlers{
FunctionProxy: httpHeaderMiddleware(proxy.NewHandlerFunc(*config, invokeResolver, false)),
DeleteFunction: httpHeaderMiddleware(handlers.MakeDeleteHandler(client, cni)),
DeployFunction: httpHeaderMiddleware(handlers.MakeDeployHandler(client, cni, baseUserSecretsPath, alwaysPull)),
FunctionLister: httpHeaderMiddleware(handlers.MakeReadHandler(client)),
FunctionStatus: httpHeaderMiddleware(handlers.MakeReplicaReaderHandler(client)),
ScaleFunction: httpHeaderMiddleware(handlers.MakeReplicaUpdateHandler(client, cni)),
UpdateFunction: httpHeaderMiddleware(handlers.MakeUpdateHandler(client, cni, baseUserSecretsPath, alwaysPull)),
Health: httpHeaderMiddleware(func(w http.ResponseWriter, r *http.Request) {}),
Info: httpHeaderMiddleware(handlers.MakeInfoHandler(faasd.Version, faasd.GitCommit)),
ListNamespaces: httpHeaderMiddleware(handlers.MakeNamespacesLister(client)),
Secrets: httpHeaderMiddleware(handlers.MakeSecretHandler(client.NamespaceService(), baseUserSecretsPath)),
Logs: httpHeaderMiddleware(logs.NewLogHandlerFunc(faasdlogs.New(), config.ReadTimeout)),
MutateNamespace: httpHeaderMiddleware(handlers.MakeMutateNamespace(client)),
}
log.Printf("Listening on: 0.0.0.0:%d", *config.TCPPort)
bootstrap.Serve(cmd.Context(), &bootstrapHandlers, config)
return nil
}
/*
* Mutiple namespace support was added after release 0.13.0
* Function will help users to migrate on multiple namespace support of faasd
*/
func moveSecretsToDefaultNamespaceSecrets(baseSecretPath string, defaultNamespace string) error {
newSecretPath := path.Join(baseSecretPath, defaultNamespace)
err := ensureSecretsDir(newSecretPath)
if err != nil {
return err
}
files, err := os.ReadDir(baseSecretPath)
if err != nil {
return err
}
for _, f := range files {
if !f.IsDir() {
newPath := path.Join(newSecretPath, f.Name())
// A non-nil error means the file wasn't found in the
// destination path
if _, err := os.Stat(newPath); err != nil {
oldPath := path.Join(baseSecretPath, f.Name())
if err := copyFile(oldPath, newPath); err != nil {
return err
}
log.Printf("[Migration] Copied %s to %s", oldPath, newPath)
}
}
}
return nil
}
func copyFile(src, dst string) error {
inputFile, err := os.Open(src)
if err != nil {
return fmt.Errorf("opening %s failed %w", src, err)
}
defer inputFile.Close()
outputFile, err := os.OpenFile(dst, os.O_CREATE|os.O_WRONLY|os.O_APPEND, secretDirPermission)
if err != nil {
return fmt.Errorf("opening %s failed %w", dst, err)
}
defer outputFile.Close()
// Changed from os.Rename due to issue in #201
if _, err := io.Copy(outputFile, inputFile); err != nil {
return fmt.Errorf("writing into %s failed %w", outputFile.Name(), err)
}
return nil
}
func httpHeaderMiddleware(next http.HandlerFunc) http.HandlerFunc {
func listNamespaces() func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-OpenFaaS-EULA", "openfaas-ce")
next.ServeHTTP(w, r)
list := []string{""}
out, _ := json.Marshal(list)
w.Write(out)
}
}

View File

@ -4,7 +4,6 @@ import (
"fmt"
"github.com/morikuni/aec"
"github.com/openfaas/faasd/pkg"
"github.com/spf13/cobra"
)
@ -23,8 +22,19 @@ func RootCommand() *cobra.Command {
return rootCommand
}
var (
// GitCommit Git Commit SHA
GitCommit string
// Version version of the CLI
Version string
)
// Execute faasd
func Execute() error {
func Execute(version, gitCommit string) error {
// Get Version and GitCommit values from main.go.
Version = version
GitCommit = gitCommit
if err := rootCommand.Execute(); err != nil {
return err
@ -36,16 +46,7 @@ var rootCommand = &cobra.Command{
Use: "faasd",
Short: "Start faasd",
Long: `
faasd Community Edition (CE):
Learn how to build, secure, and monitor functions with faasd with
the eBook:
https://openfaas.gumroad.com/l/serverless-for-everyone-else
License: OpenFaaS CE EULA with faasd addendum:
https://github.com/openfaas/faasd/blob/master/EULA.md
faasd - serverless without Kubernetes
`,
RunE: runRootCommand,
SilenceUsage: true,
@ -72,7 +73,7 @@ func parseBaseCommand(_ *cobra.Command, _ []string) {
}
func printVersion() {
fmt.Printf("faasd Community Edition (CE) version: %s\tcommit: %s\n", pkg.GetVersion(), pkg.GitCommit)
fmt.Printf("faasd version: %s\tcommit: %s\n", GetVersion(), GitCommit)
}
func printLogo() {
@ -80,6 +81,14 @@ func printLogo() {
fmt.Println(logoText)
}
// GetVersion get latest version
func GetVersion() string {
if len(Version) == 0 {
return "dev"
}
return Version
}
// Logo for version and root command
const Logo = ` __ _
/ _| __ _ __ _ ___ __| |

View File

@ -2,10 +2,12 @@ package cmd
import (
"fmt"
"io/ioutil"
"log"
"os"
"os/signal"
"path"
"strings"
"sync"
"syscall"
"time"
@ -15,7 +17,6 @@ import (
"github.com/spf13/cobra"
flag "github.com/spf13/pflag"
units "github.com/docker/go-units"
"github.com/openfaas/faasd/pkg"
)
@ -38,10 +39,9 @@ func init() {
}
var upCmd = &cobra.Command{
Use: "up",
Short: "Start faasd",
RunE: runUp,
PreRunE: preRunE,
Use: "up",
Short: "Start faasd",
RunE: runUp,
}
func runUp(cmd *cobra.Command, _ []string) error {
@ -69,18 +69,23 @@ func runUp(cmd *cobra.Command, _ []string) error {
return err
}
log.Printf("Supervisor created in: %s\n", units.HumanDuration(time.Since(start)))
log.Printf("Supervisor created in: %s\n", time.Since(start).String())
start = time.Now()
if err := supervisor.Start(services); err != nil {
err = supervisor.Start(services)
if err != nil {
return err
}
defer supervisor.Close()
log.Printf("Supervisor init done in: %s\n", units.HumanDuration(time.Since(start)))
log.Printf("Supervisor init done in: %s\n", time.Since(start).String())
shutdownTimeout := time.Second * 1
timeout := time.Second * 60
proxyDoneCh := make(chan bool)
wg := sync.WaitGroup{}
wg.Add(1)
@ -97,38 +102,38 @@ func runUp(cmd *cobra.Command, _ []string) error {
fmt.Println(err)
}
// TODO: close proxies
// Close proxy
proxyDoneCh <- true
time.AfterFunc(shutdownTimeout, func() {
wg.Done()
})
}()
localResolver := pkg.NewLocalResolver(path.Join(cfg.workingDir, "hosts"))
go localResolver.Start()
gatewayURLChan := make(chan string, 1)
proxyPort := 8080
proxy := pkg.NewProxy(proxyPort, timeout)
go proxy.Start(gatewayURLChan, proxyDoneCh)
proxies := map[uint32]*pkg.Proxy{}
for _, svc := range services {
for _, port := range svc.Ports {
go func() {
time.Sleep(3 * time.Second)
listenPort := port.Port
if _, ok := proxies[listenPort]; ok {
return fmt.Errorf("port %d already allocated", listenPort)
}
hostIP := "0.0.0.0"
if len(port.HostIP) > 0 {
hostIP = port.HostIP
}
upstream := fmt.Sprintf("%s:%d", svc.Name, port.TargetPort)
proxies[listenPort] = pkg.NewProxy(upstream, listenPort, hostIP, timeout, localResolver)
fileData, fileErr := ioutil.ReadFile(path.Join(cfg.workingDir, "hosts"))
if fileErr != nil {
log.Println(fileErr)
return
}
}
// TODO: track proxies for later cancellation when receiving sigint/term
for _, v := range proxies {
go v.Start()
}
host := ""
lines := strings.Split(string(fileData), "\n")
for _, line := range lines {
if strings.Index(line, "gateway") > -1 {
host = line[:strings.Index(line, "\t")]
}
}
log.Printf("[up] Sending %s to proxy\n", host)
gatewayURLChan <- host + ":8080"
close(gatewayURLChan)
}()
wg.Wait()
return nil
@ -166,7 +171,7 @@ func makeFile(filePath, fileContents string) error {
return nil
} else if os.IsNotExist(err) {
log.Printf("Writing to: %q\n", filePath)
return os.WriteFile(filePath, []byte(fileContents), workingDirectoryPermission)
return ioutil.WriteFile(filePath, []byte(fileContents), workingDirectoryPermission)
} else {
return err
}
@ -204,11 +209,3 @@ func parseUpFlags(cmd *cobra.Command) (upConfig, error) {
parsed.workingDir = faasdwd
return parsed, err
}
func preRunE(cmd *cobra.Command, _ []string) error {
if err := pkg.ConnectivityCheck(); err != nil {
return fmt.Errorf("the OpenFaaS CE EULA requires Internet access, upgrade to faasd Pro to continue")
}
return nil
}

View File

@ -1,45 +1,43 @@
version: "3.7"
services:
basic-auth-plugin:
image: "docker.io/openfaas/basic-auth-plugin:0.18.17${ARCH_SUFFIX}"
environment:
- port=8080
- secret_mount_path=/run/secrets
- user_filename=basic-auth-user
- pass_filename=basic-auth-password
volumes:
# we assume cwd == /var/lib/faasd
- type: bind
source: ./secrets/basic-auth-password
target: /run/secrets/basic-auth-password
- type: bind
source: ./secrets/basic-auth-user
target: /run/secrets/basic-auth-user
cap_add:
- CAP_NET_RAW
nats:
image: docker.io/library/nats-streaming:0.25.6
# nobody
user: "65534"
image: docker.io/library/nats-streaming:0.11.2
command:
- "/nats-streaming-server"
- "-m"
- "8222"
- "--store=file"
- "--dir=/nats"
- "--store=memory"
- "--cluster_id=faas-cluster"
volumes:
# Data directory
- type: bind
source: ./nats
target: /nats
# ports:
# - "127.0.0.1:8222:8222"
prometheus:
image: docker.io/prom/prometheus:v3.1.0
# nobody
user: "65534"
image: docker.io/prom/prometheus:v2.14.0
volumes:
# Config directory
- type: bind
source: ./prometheus.yml
target: /etc/prometheus/prometheus.yml
# Data directory
- type: bind
source: ./prometheus
target: /prometheus
cap_add:
- CAP_NET_RAW
ports:
- "127.0.0.1:9090:9090"
gateway:
image: ghcr.io/openfaas/gateway:0.27.12
image: "docker.io/openfaas/gateway:0.18.17${ARCH_SUFFIX}"
environment:
- basic_auth=true
- functions_provider_url=http://faasd-provider:8081/
@ -49,9 +47,10 @@ services:
- upstream_timeout=65s
- faas_nats_address=nats
- faas_nats_port=4222
- auth_proxy_url=http://basic-auth-plugin:8080/validate
- auth_proxy_pass_body=false
- secret_mount_path=/run/secrets
- scale_from_zero=true
- function_namespace=openfaas-fn
volumes:
# we assume cwd == /var/lib/faasd
- type: bind
@ -63,13 +62,12 @@ services:
cap_add:
- CAP_NET_RAW
depends_on:
- basic-auth-plugin
- nats
- prometheus
ports:
- "8080:8080"
queue-worker:
image: ghcr.io/openfaas/queue-worker:0.14.2
image: docker.io/openfaas/queue-worker:0.11.2
environment:
- faas_nats_address=nats
- faas_nats_port=4222

View File

@ -1,15 +1,9 @@
## Instructions for building and testing faasd locally
## Manual installation of faasd for development
> Note: if you're just wanting to try out faasd, then it's likely that you're on the wrong page. This is a detailed set of instructions for those wanting to contribute or customise faasd. Feel free to go back to the homepage and pick a tutorial instead.
Do you want to help the community test a pull request?
See these instructions instead: [Testing patches](/docs/PATCHES.md)
### Pre-reqs
> It's recommended that you do not install Docker on the same host as faasd, since 1) they may both use different versions of containerd and 2) docker's networking rules can disrupt faasd's networking. When using faasd - make your faasd server a faasd server, and build container image on your laptop or in a CI pipeline.
* Linux
PC / Cloud - any Linux that containerd works on should be fair game, but faasd is tested with Ubuntu 18.04
@ -20,15 +14,15 @@ See these instructions instead: [Testing patches](/docs/PATCHES.md)
For Windows users, install [Git Bash](https://git-scm.com/downloads) along with multipass or vagrant. You can also use WSL1 or WSL2 which provides a Linux environment.
You will also need [containerd](https://github.com/containerd/containerd) and the [CNI plugins](https://github.com/containernetworking/plugins)
You will also need [containerd v1.3.5](https://github.com/containerd/containerd) and the [CNI plugins v0.8.5](https://github.com/containernetworking/plugins)
[faas-cli](https://github.com/openfaas/faas-cli) is optional, but recommended.
If you're using multipass, then allocate sufficient resources:
```bash
```sh
multipass launch \
--memory 4G \
--mem 4G \
-c 2 \
-n faasd
@ -38,7 +32,7 @@ multipass shell faasd
### Get runc
```bash
```sh
sudo apt update \
&& sudo apt install -qy \
runc \
@ -57,13 +51,14 @@ curl -sLS https://cli.openfaas.com | sudo sh
#### Install the CNI plugins:
* For PC run `export ARCH=amd64`
* For RPi/armhf run `export ARCH=arm`
* For arm64 run `export ARCH=arm64`
Then run:
```bash
```sh
export ARCH=amd64
export CNI_VERSION=v0.9.1
export CNI_VERSION=v0.8.5
sudo mkdir -p /opt/cni/bin
curl -sSL https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-linux-${ARCH}-${CNI_VERSION}.tgz | sudo tar -xz -C /opt/cni/bin
@ -82,27 +77,36 @@ EOF'
### Get containerd
You have three options - binaries for PC, binaries for armhf, or build from source.
* Install containerd `x86_64` only
```bash
export VER=1.7.22
curl -sSL https://github.com/containerd/containerd/releases/download/v$VER/containerd-$VER-linux-amd64.tar.gz -o /tmp/containerd.tar.gz \
```sh
export VER=1.3.5
curl -sSL https://github.com/containerd/containerd/releases/download/v$VER/containerd-$VER-linux-amd64.tar.gz > /tmp/containerd.tar.gz \
&& sudo tar -xvf /tmp/containerd.tar.gz -C /usr/local/bin/ --strip-components=1
containerd -version
```
* Or get my containerd binaries for Raspberry Pi (armhf)
Building `containerd` on armhf is extremely slow, so I've provided binaries for you.
```sh
curl -sSL https://github.com/alexellis/containerd-armhf/releases/download/v1.3.5/containerd.tgz | sudo tar -xvz --strip-components=2 -C /usr/local/bin/
```
* Or clone / build / install [containerd](https://github.com/containerd/containerd) from source:
```bash
```sh
export GOPATH=$HOME/go/
mkdir -p $GOPATH/src/github.com/containerd
cd $GOPATH/src/github.com/containerd
git clone https://github.com/containerd/containerd
cd containerd
git fetch origin --tags
git checkout v1.7.22
git checkout v1.3.5
make
sudo make install
@ -112,8 +116,8 @@ containerd -version
#### Ensure containerd is running
```bash
curl -sLS https://raw.githubusercontent.com/containerd/containerd/v1.7.22/containerd.service > /tmp/containerd.service
```sh
curl -sLS https://raw.githubusercontent.com/containerd/containerd/v1.3.5/containerd.service > /tmp/containerd.service
# Extend the timeouts for low-performance VMs
echo "[Manager]" | tee -a /tmp/containerd.service
@ -128,7 +132,7 @@ sudo systemctl restart containerd
Or run ad-hoc. This step can be useful for exploring why containerd might fail to start.
```bash
```sh
sudo containerd &
```
@ -136,13 +140,13 @@ sudo containerd &
> This is required to allow containers in containerd to access the Internet via your computer's primary network interface.
```bash
```sh
sudo /sbin/sysctl -w net.ipv4.conf.all.forwarding=1
```
Make the setting permanent:
```bash
```sh
echo "net.ipv4.conf.all.forwarding=1" | sudo tee -a /etc/sysctl.conf
```
@ -150,7 +154,7 @@ echo "net.ipv4.conf.all.forwarding=1" | sudo tee -a /etc/sysctl.conf
#### Get build packages
```bash
```sh
sudo apt update \
&& sudo apt install -qy \
runc \
@ -162,8 +166,8 @@ You may find alternative package names for CentOS and other Linux distributions.
#### Install Go 1.13 (x86_64)
```bash
curl -SLf https://golang.org/dl/go1.16.linux-amd64.tar.gz > /tmp/go.tgz
```sh
curl -sSLf https://dl.google.com/go/go1.13.6.linux-amd64.tar.gz > /tmp/go.tgz
sudo rm -rf /usr/local/go/
sudo mkdir -p /usr/local/go/
sudo tar -xvf /tmp/go.tgz -C /usr/local/go/ --strip-components=1
@ -176,15 +180,15 @@ go version
You should also add the following to `~/.bash_profile`:
```bash
```sh
echo "export GOPATH=\$HOME/go/" | tee -a $HOME/.bash_profile
echo "export PATH=\$PATH:/usr/local/go/bin/" | tee -a $HOME/.bash_profile
```
#### Or on Raspberry Pi (armhf)
```bash
curl -SLsf https://golang.org/dl/go1.16.linux-armv6l.tar.gz > go.tgz
```sh
curl -SLsf https://dl.google.com/go/go1.13.6.linux-armv6l.tar.gz > go.tgz
sudo rm -rf /usr/local/go/
sudo mkdir -p /usr/local/go/
sudo tar -xvf go.tgz -C /usr/local/go/ --strip-components=1
@ -197,7 +201,7 @@ go version
#### Clone faasd and its systemd unit files
```bash
```sh
mkdir -p $GOPATH/src/github.com/openfaas/
cd $GOPATH/src/github.com/openfaas/
git clone https://github.com/openfaas/faasd
@ -205,7 +209,7 @@ git clone https://github.com/openfaas/faasd
#### Build `faasd` from source (optional)
```bash
```sh
cd $GOPATH/src/github.com/openfaas/faasd
cd faasd
make local
@ -216,15 +220,18 @@ sudo cp bin/faasd /usr/local/bin
#### Or, download and run `faasd` (binaries)
```bash
```sh
# For x86_64
export SUFFIX=""
# armhf
export SUFFIX="-armhf"
# arm64
export SUFFIX="-arm64"
# Then download
curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.16.2/faasd$SUFFIX" \
curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.8.2/faasd$SUFFIX" \
-o "/tmp/faasd" \
&& chmod +x "/tmp/faasd"
sudo mv /tmp/faasd /usr/local/bin/
@ -234,26 +241,26 @@ sudo mv /tmp/faasd /usr/local/bin/
This step installs faasd as a systemd unit file, creates files in `/var/lib/faasd`, and writes out networking configuration for the CNI bridge networking plugin.
```bash
```sh
sudo faasd install
2020/02/17 17:38:06 Writing to: "/var/lib/faasd/secrets/basic-auth-password"
2020/02/17 17:38:06 Writing to: "/var/lib/faasd/secrets/basic-auth-user"
Login with:
sudo -E cat /var/lib/faasd/secrets/basic-auth-password | faas-cli login -s
sudo cat /var/lib/faasd/secrets/basic-auth-password | faas-cli login -s
```
You can now log in either from this machine or a remote machine using the OpenFaaS UI, or CLI.
Check that faasd is ready:
```bash
```
sudo journalctl -u faasd
```
You should see output like:
```bash
```
Feb 17 17:46:35 gold-survive faasd[4140]: 2020/02/17 17:46:35 Starting faasd proxy on 8080
Feb 17 17:46:35 gold-survive faasd[4140]: Gateway: 10.62.0.5:8080
Feb 17 17:46:35 gold-survive faasd[4140]: 2020/02/17 17:46:35 [proxy] Wait for done
@ -262,7 +269,7 @@ Feb 17 17:46:35 gold-survive faasd[4140]: 2020/02/17 17:46:35 [proxy] Begin list
To get the CLI for the command above run:
```bash
```sh
curl -sSLf https://cli.openfaas.com | sudo sh
```
@ -318,7 +325,7 @@ faasd provider
Look in `hosts` in the current working folder or in `/var/lib/faasd/` to get the IP for the gateway or Prometheus
```bash
```sh
127.0.0.1 localhost
10.62.0.1 faasd-provider
@ -348,31 +355,3 @@ The default Basic Auth username is `admin`, which is written to `/var/lib/faasd/
* `faasd install` - install faasd and containerd with systemd, this must be run from `$GOPATH/src/github.com/openfaas/faasd`
* `journalctl -u faasd -f` - faasd service logs
* `journalctl -u faasd-provider -f` - faasd-provider service logs
#### Uninstall
* Stop faasd and faasd-provider
```
sudo systemctl stop faasd
sudo systemctl stop faasd-provider
sudo systemctl stop containerd
```
* Remove faasd from machine
```
sudo systemctl disable faasd
sudo systemctl disable faasd-provider
sudo systemctl disable containerd
sudo rm -rf /usr/local/bin/faasd
sudo rm -rf /var/lib/faasd
sudo rm -rf /usr/lib/systemd/system/faasd-provider.service
sudo rm -rf /usr/lib/systemd/system/faasd.service
sudo rm -rf /usr/lib/systemd/system/containerd
sudo systemctl daemon-reload
```
* Remove additional dependencies. Be cautious as other software will be dependent on these.
```
sudo apt-get remove runc bridge-utils
sudo rm -rf /opt/cni/bin
```

View File

@ -27,112 +27,111 @@ It took me about 2-3 minutes to run through everything after installing multipas
* Get my cloud-config.txt file
```sh
curl -sSLO https://raw.githubusercontent.com/openfaas/faasd/master/cloud-config.txt
```
```sh
curl -sSLO https://raw.githubusercontent.com/openfaas/faasd/master/cloud-config.txt
```
* Boot the VM
* Update the SSH key to match your own, edit `cloud-config.txt`:
The `cloud-config.txt` contains an ssh key to allow your local machine to access the VM. However, this must be updated with your local ssh key.
This command will update the key with your local public key value and start the VM.
Replace the 2nd line with the contents of `~/.ssh/id_rsa.pub`:
```sh
sed "s/ssh-rsa.*/$(cat $HOME/.ssh/id_*.pub)/" cloud-config.txt | multipass launch --name faasd --cloud-init -
```
```
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8Q/aUYUr3P1XKVucnO9mlWxOjJm+K01lHJR90MkHC9zbfTqlp8P7C3J26zKAuzHXOeF+VFxETRr6YedQKW9zp5oP7sN+F2gr/pO7GV3VmOqHMV7uKfyUQfq7H1aVzLfCcI7FwN2Zekv3yB7kj35pbsMa1Za58aF6oHRctZU6UWgXXbRxP+B04DoVU7jTstQ4GMoOCaqYhgPHyjEAS3DW0kkPW6HzsvJHkxvVcVlZ/wNJa1Ie/yGpzOzWIN0Ol0t2QT/RSWOhfzO1A2P0XbPuZ04NmriBonO9zR7T1fMNmmtTuK7WazKjQT3inmYRAqU6pe8wfX8WIWNV7OowUjUsv alex@alexr.local
```
This can also be done manually, just replace the 2nd line of the `cloud-config.txt` with the contents of your public ssh key, usually either `~/.ssh/id_rsa.pub` or `~/.ssh/id_ed25519.pub`
* Boot the VM
```
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8Q/aUYUr3P1XKVucnO9mlWxOjJm+K01lHJR90MkHC9zbfTqlp8P7C3J26zKAuzHXOeF+VFxETRr6YedQKW9zp5oP7sN+F2gr/pO7GV3VmOqHMV7uKfyUQfq7H1aVzLfCcI7FwN2Zekv3yB7kj35pbsMa1Za58aF6oHRctZU6UWgXXbRxP+B04DoVU7jTstQ4GMoOCaqYhgPHyjEAS3DW0kkPW6HzsvJHkxvVcVlZ/wNJa1Ie/yGpzOzWIN0Ol0t2QT/RSWOhfzO1A2P0XbPuZ04NmriBonO9zR7T1fMNmmtTuK7WazKjQT3inmYRAqU6pe8wfX8WIWNV7OowUjUsv alex@alexr.local
```
```sh
multipass launch --cloud-init cloud-config.txt --name faasd
```
* Get the VM's IP and connect with `ssh`
```sh
multipass info faasd
Name: faasd
State: Running
IPv4: 192.168.64.14
Release: Ubuntu 18.04.3 LTS
Image hash: a720c34066dc (Ubuntu 18.04 LTS)
Load: 0.79 0.19 0.06
Disk usage: 1.1G out of 4.7G
Memory usage: 145.6M out of 985.7M
```
```sh
multipass info faasd
Name: faasd
State: Running
IPv4: 192.168.64.14
Release: Ubuntu 18.04.3 LTS
Image hash: a720c34066dc (Ubuntu 18.04 LTS)
Load: 0.79 0.19 0.06
Disk usage: 1.1G out of 4.7G
Memory usage: 145.6M out of 985.7M
```
Set the variable `IP`:
Set the variable `IP`:
```
export IP="192.168.64.14"
```
```
export IP="192.168.64.14"
```
You can also try to use `jq` to get the IP into a variable:
You can also try to use `jq` to get the IP into a variable:
```sh
export IP=$(multipass info faasd --format json| jq -r '.info.faasd.ipv4[0]')
```
```sh
export IP=$(multipass info faasd --format json| jq '.info.faasd.ipv4[0]' | tr -d '\"')
```
Connect to the IP listed:
Connect to the IP listed:
```sh
ssh ubuntu@$IP
```
```sh
ssh ubuntu@$IP
```
Log out once you know it works.
Log out once you know it works.
* Let's capture the authentication password into a file for use with `faas-cli`
```
ssh ubuntu@$IP "sudo cat /var/lib/faasd/secrets/basic-auth-password" > basic-auth-password
```
```
ssh ubuntu@192.168.64.14 "sudo cat /var/lib/faasd/secrets/basic-auth-password" > basic-auth-password
```
## Try faasd (OpenFaaS)
* Login from your laptop (the host)
```
export OPENFAAS_URL=http://$IP:8080
cat basic-auth-password | faas-cli login -s
```
```
export OPENFAAS_URL=http://$IP:8080
cat basic-auth-password | faas-cli login -s
```
* Deploy a function and invoke it
```
faas-cli store deploy figlet --env write_timeout=1s
echo "faasd" | faas-cli invoke figlet
```
faas-cli store deploy figlet --env write_timeout=1s
echo "faasd" | faas-cli invoke figlet
faas-cli describe figlet
faas-cli describe figlet
# Run async
curl -i -d "faasd-async" $OPENFAAS_URL/async-function/figlet
# Run async
curl -i -d "faasd-async" $OPENFAAS_URL/async-function/figlet
# Run async with a callback
# Run async with a callback
curl -i -d "faasd-async" -H "X-Callback-Url: http://some-request-bin.com/path" $OPENFAAS_URL/async-function/figlet
```
curl -i -d "faasd-async" -H "X-Callback-Url: http://some-request-bin.com/path" $OPENFAAS_URL/async-function/figlet
```
You can also checkout the other store functions: `faas-cli store list`
You can also checkout the other store functions: `faas-cli store list`
* Try the UI
Head over to the UI from your laptop and remember that your password is in the `basic-auth-password` file. The username is `admin`:
Head over to the UI from your laptop and remember that your password is in the `basic-auth-password` file. The username is `admin.:
```
echo http://$IP:8080
```
```
echo http://$IP:8080
```
* Stop/start the instance
```sh
multipass stop faasd
```
```sh
multipass stop faasd
```
* Delete, if you want to:
```
multipass delete --purge faasd
```
```
multipass delete --purge faasd
```
You now have a faasd appliance on your Mac. You can also use this cloud-init file with public cloud like AWS or DigitalOcean.

View File

@ -1,88 +0,0 @@
## Instructions for testing a patch for faasd
### Launch a virtual machine
You can use any kind of Linux virtual machine, Ubuntu 20.04 is recommended.
Launch a cloud VM or use [Multipass](https://multipass.run), which is free to use an can be run locally. A Raspberry Pi 3 or 4 could also be used, but will need you to run `make dist` to cross compile a valid binary.
### Copy over your SSH key
Your SSH key will be used, so that you can copy a new faasd binary over to the host.
```bash
multipass launch \
--memory 4G \
-c 2 \
-n faasd
# Then access its shell
multipass shell faasd
# Edit .ssh/authorized_keys
# Add .ssh/id_rsa.pub from your host and save the file
```
### Install faasd on the VM
You start off with the upstream version of faasd on the host, then add the new version over the top later on.
```bash
cd /tmp/
git clone https://github.com/openfaas/faasd --depth=1
cd faasd/hack
./install.sh
# Run the login command given to you at the end of the script
```
Get the multipass IP address:
```bash
export IP=$(multipass info faasd --format json| jq -r '.info.faasd.ipv4[0]')
```
### Build a new faasd binary with the patch
Check out faasd on your local computer
```bash
git clone https://github.com/openfaas/faasd
cd faasd
gh pr checkout #PR_NUMBER_HERE
GOOS=linux go build
# You can also run "make dist" which is slower, but includes
# a version and binaries for other platforms such as the Raspberry Pi
```
### Copy it over to the VM
Now build a new faasd binary and copy it to the VM:
```bash
scp faasd ubuntu@$IP:~/
```
Now deploy the new version on the VM:
```bash
killall -9 faasd-linux; killall -9 faasd-linux ; mv ./faasd-linux /usr/local/bin/faasd
```
### Check it worked and test that patch
Now run a command with `faas-cli` such as:
* `faas-cli list`
* `faas-cli version`
See the testing instructions on the PR and run through those steps.
Post your results on GitHub to assist the creator of the pull request.
You can see how to get the logs for various components using the [eBook Serverless For Everyone Else](https://openfaas.gumroad.com/l/serverless-for-everyone-else), or by consulting the [DEV.md](/docs/DEV.md) guide.

View File

@ -1,7 +0,0 @@
# Documentation
- [Develop faasd](./DEV.md) - Instructions for building and testing faasd locally
- [Testing faasd](./PATCHES.md) - Instructions for testing a patch for faasd
- [Roadmap](./ROADMAP.md) - Overview of features, backlog and known issues
- [Run faasd with multipass](./MULTIPASS.md) - Tutorial on how to run faasd on a local multipass VM
- [Terraform modules](./TERRAFORM.md) - A collection of official and community provided terraform modules for faasd

View File

@ -1,151 +0,0 @@
# faasd backlog and features
It's important to understand the vision for faasd vs OpenFaaS CE/Pro.
faasd is a single-node implementation of OpenFaaS.
It is supposed to be a lightweight, low-overhead, way to deploy OpenFaaS functions for functions which do not need planet-scale.
It is not supposed to have multiple replicas, clustering, HA, or auto-scaling.
[Learn when to use faasd](https://docs.openfaas.com/deployment/)
## Supported operations
* `faas-cli login`
* `faas-cli up`
* `faas-cli list`
* `faas-cli describe`
* `faas-cli deploy --update=true --replace=false`
* `faas-cli invoke --async`
* `faas-cli invoke`
* `faas-cli rm`
* `faas-cli store list/deploy/inspect`
* `faas-cli version`
* `faas-cli namespace`
* `faas-cli secret`
* `faas-cli logs`
* `faas-cli auth` - supported for Basic Authentication and OpenFaaS Pro with OIDC and Single-sign On.
The OpenFaaS REST API is supported by faasd, learn more in the [manual](https://store.openfaas.com/l/serverless-for-everyone-else) under "Can I get an API with that?"
## Constraints vs OpenFaaS on Kubernetes
faasd suits certain use-cases as mentioned in the [README.md](/README.md) file, for those who want a solution which can scale out horizontally with minimum effort, Kubernetes or K3s is a valid option.
Which is right for you? [Read a comparison in the OpenFaaS docs](https://docs.openfaas.com/deployment/)
### One replica per function
Functions only support one replica for each function, so that means horizontal scaling is not available.
It can scale vertically, and this may be a suitable alternative for many use-cases. See the [YAML reference for how to configure limits](https://docs.openfaas.com/reference/yaml/).
Workaround: deploy multiple, dynamically named functions `scraper-1`, `scraper-2`, `scraper-3` and set up a reverse proxy rule to load balance i.e. `scraper.example.com => [/function/scraper-1, /function/scraper-2, /function/scraper-3]`.
### Scale from zero may give a non-200
faasd itself does not implement a health check to determine if a function is ready for traffic. Since faasd doesn't support auto-scaling, this is unlikely to affect you.
Workaround: Have your client retry HTTP calls, or don't scale to zero.
### Leaf-node only - no clustering
faasd is operates on a leaf-node/single-node model. If this is an issue for you, but you have resource constraints, you will need to use [OpenFaaS CE or Pro on Kubernetes](https://docs.openfaas.com/deployment/).
There are no plans to add any form of clustering or multi-node support to faasd.
See past discussion at: [HA / resilience in faasd #225](https://github.com/openfaas/faasd/issues/225)
What about HA and fault tolerance?
To achieve fault tolerance, you could put two faasd instances behind a load balancer or proxy, but you will need to deploy the same set of functions to each.
An alternative would be to take regular VM backups or snapshots.
### No rolling updates are available today
When running `faas-cli deploy`, your old function is removed before the new one is started. This may cause a period of downtime, depending on the timeouts and grace periods you set.
Workaround: deploy uniquely named functions i.e. `scraper-1` and `scraper-2` with a reverse proxy rule that maps `/function/scraper` to the active version.
## Known issues
### Troubleshooting
There is a very detailed chapter on troubleshooting in the eBook [Serverless For Everyone Else](https://store.openfaas.com/l/serverless-for-everyone-else)
### Your function timed-out at 60 seconds
This is no longer an issue, see the manual for how to configure a longer timeout, updated 3rd October 2022.
### Non 200 HTTP status from the gateway upon reboot
This issue appears to happen sporadically and only for some users.
If you get a non 200 HTTP code from the gateway, or caddy after installing faasd, check the logs of faasd:
```bash
sudo journalctl -u faasd
```
If you see the following error:
```
unable to dial to 10.62.0.5:8080, error: dial tcp 10.62.0.5:8080: connect: no route to host
```
Restart the faasd service with:
```bash
sudo systemctl restart faasd
```
## Backlog
Should have:
* [ ] Restart any of the containers in docker-compose.yaml if they crash.
* [ ] Asynchronous function deployment and deletion (currently synchronous/blocking)
Nice to Have:
* [ ] Live rolling-updates, with zero downtime (may require using IDs instead of names for function containers)
* [ ] Apply a total memory limit for the host (if a node has 1GB of RAM, don't allow more than 1GB of RAM to be specified in the limits field)
* [ ] Terraform for AWS EC2
Won't have:
* [ ] Clustering
* [ ] Multiple replicas per function
### Completed
* [x] Docs or examples on how to use the various event connectors (Yes in the eBook)
* [x] Resolve core services from functions by populating/sharing `/etc/hosts` between `faasd` and `faasd-provider`
* [x] Provide a cloud-init configuration for faasd bootstrap
* [x] Configure core services from a docker-compose.yaml file
* [x] Store and fetch logs from the journal
* [x] Add support for using container images in third-party public registries
* [x] Add support for using container images in private third-party registries
* [x] Provide a cloud-config.txt file for automated deployments of `faasd`
* [x] Inject / manage IPs between core components for service to service communication - i.e. so Prometheus can scrape the OpenFaaS gateway - done via `/etc/hosts` mount
* [x] Add queue-worker and NATS
* [x] Create faasd.service and faasd-provider.service
* [x] Self-install / create systemd service via `faasd install`
* [x] Restart containers upon restart of faasd
* [x] Clear / remove containers and tasks with SIGTERM / SIGINT
* [x] Determine arm64 containers to run for gateway
* [x] Configure `basic_auth` to protect the OpenFaaS gateway and faasd-provider HTTP API
* [x] Setup custom working directory for faasd `/var/lib/faasd/`
* [x] Use CNI to create network namespaces and adapters
* [x] Optionally expose core services from the docker-compose.yaml file, locally or to all adapters.
* [x] ~~[containerd can't pull image from Github Docker Package Registry](https://github.com/containerd/containerd/issues/3291)~~ ghcr.io support
* [x] Provide [simple Caddyfile example](https://blog.alexellis.io/https-inlets-local-endpoints/) in the README showing how to expose the faasd proxy on port 80/443 with TLS
* [x] Annotation support
* [x] Hard memory limits for functions
* [x] Terraform for DigitalOcean
* [x] [Store and retrieve annotations in function spec](https://github.com/openfaas/faasd/pull/86) - in progress
* [x] An installer for faasd and dependencies - runc, containerd
* [x] Offer a recommendation or implement a strategy for faasd replication/HA
* [x] [Remove / deprecate armhf / armv7 support](https://github.com/openfaas/faasd/issues/364)

View File

@ -1,14 +0,0 @@
# Terraform Modules for faasd
| Terraform Module | Author | Origin | Status |
|-----------|--------|--------|--------|
| [faasd on DigitalOcean](https://github.com/openfaas/faasd/tree/master/docs/bootstrap/digitalocean-terraform) | OpenFaaS | Official | Active |
| [faasd on DigitalOcean](https://github.com/jsiebens/terraform-digitalocean-faasd) | Johan Siebens | Community | Active |
| [faasd on Google Cloud Platform](https://github.com/jsiebens/terraform-google-faasd) | Johan Siebens | Community | Active |
| [faasd on Microsoft Azure ](https://github.com/jsiebens/terraform-azurerm-faasd) | Johan Siebens | Community | Active |
| [faasd on Equinix Metal](https://github.com/jsiebens/terraform-equinix-faasd) | Johan Siebens | Community | Active |
| [faasd on Scaleway](https://github.com/jsiebens/terraform-scaleway-faasd) | Johan Siebens | Community | Active |
| [faasd on Amazon Web Services](https://github.com/jsiebens/terraform-aws-faasd) |Johan Siebens | Community | Active |
| [faasd on Linode](https://github.com/itTrident/terraform-linode-faasd) | itTrident | Community | Active |
| [faasd on Vultr](https://github.com/itTrident/terraform-vultr-faasd) | itTrident | Community | Active |
| [faasd on Exoscale](https://github.com/itTrident/terraform-exoscale-faasd) | itTrident | Community | Active |

3
docs/bootstrap/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
/.terraform/
/terraform.tfstate
/terraform.tfstate.backup

20
docs/bootstrap/README.md Normal file
View File

@ -0,0 +1,20 @@
# Bootstrap faasd on Digitalocean
1) [Sign up to DigitalOcean](https://www.digitalocean.com/?refcode=2962aa9e56a1&utm_campaign=Referral_Invite&utm_medium=Referral_Program&utm_source=CopyPaste)
2) [Download Terraform](https://www.terraform.io)
3) Clone this gist using the URL from the address bar
4) Run `terraform init`
5) Run `terraform apply -var="do_token=$(cat $HOME/digitalocean-access-token)"`
6) View the output for the login command and gateway URL i.e.
```
gateway_url = http://178.128.39.201:8080/
login_cmd = faas-cli login -g http://178.128.39.201:8080/ -p rvIU49CEcFcHmqxj
password = rvIU49CEcFcHmqxj
```
Note that the user-data may take a couple of minutes to come up since it will be pulling in various components and preparing the machine.
A single host with 1GB of RAM will be deployed for you, to remove at a later date simply use `terraform destroy`.
If required, you can remove the VM via `terraform destroy -var="do_token=$(cat $HOME/digitalocean-access-token)"`

View File

@ -1,23 +1,26 @@
#cloud-config
ssh_authorized_keys:
## Note: Replace with your own public key
- ${ssh_key}
package_update: true
packages:
- runc
- git
runcmd:
- curl -sLSf https://github.com/containerd/containerd/releases/download/v1.7.0/containerd-1.7.0-linux-amd64.tar.gz > /tmp/containerd.tar.gz && tar -xvf /tmp/containerd.tar.gz -C /usr/local/bin/ --strip-components=1
- curl -SLfs https://raw.githubusercontent.com/containerd/containerd/v1.7.0/containerd.service | tee /etc/systemd/system/containerd.service
- curl -sLSf https://github.com/containerd/containerd/releases/download/v1.3.5/containerd-1.3.5-linux-amd64.tar.gz > /tmp/containerd.tar.gz && tar -xvf /tmp/containerd.tar.gz -C /usr/local/bin/ --strip-components=1
- curl -SLfs https://raw.githubusercontent.com/containerd/containerd/v1.3.5/containerd.service | tee /etc/systemd/system/containerd.service
- systemctl daemon-reload && systemctl start containerd
- /sbin/sysctl -w net.ipv4.conf.all.forwarding=1
- mkdir -p /opt/cni/bin
- curl -sSL https://github.com/containernetworking/plugins/releases/download/v0.9.1/cni-plugins-linux-amd64-v0.9.1.tgz | tar -xz -C /opt/cni/bin
- curl -sSL https://github.com/containernetworking/plugins/releases/download/v0.8.5/cni-plugins-linux-amd64-v0.8.5.tgz | tar -xz -C /opt/cni/bin
- mkdir -p /go/src/github.com/openfaas/
- mkdir -p /var/lib/faasd/secrets/
- echo ${gw_password} > /var/lib/faasd/secrets/basic-auth-password
- echo admin > /var/lib/faasd/secrets/basic-auth-user
- cd /go/src/github.com/openfaas/ && git clone --depth 1 --branch 0.16.2 https://github.com/openfaas/faasd
- curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.16.2/faasd" --output "/usr/local/bin/faasd" && chmod a+x "/usr/local/bin/faasd"
- cd /go/src/github.com/openfaas/ && git clone https://github.com/openfaas/faasd && cd faasd && git checkout 0.9.2
- curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.9.2/faasd" --output "/usr/local/bin/faasd" && chmod a+x "/usr/local/bin/faasd"
- cd /go/src/github.com/openfaas/faasd/ && /usr/local/bin/faasd install
- systemctl status -l containerd --no-pager
- journalctl -u faasd-provider --no-pager

View File

@ -1,66 +0,0 @@
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/digitalocean/digitalocean" {
version = "2.11.0"
constraints = "2.11.0"
hashes = [
"h1:/qAnTOSP5KeZkF7wqLai34SKAs7aefulcUA3I8R7rRg=",
"h1:PbXtjUfvxwmkycJ0Y9Dyn66Arrpk5L8/P381SXMx2O0=",
"h1:lXLX9tmuxV7azTHd0xB0FAVrxyfBtotIz5LEJp8YUk0=",
"zh:2191adc79bdfdb3b733e0619e4f391ae91c1631c5dafda42dab561d943651fa4",
"zh:21a4f67e42dcdc10fbd7f8579247594844d09a469a3a54862d565913e4d6121d",
"zh:557d98325fafcf2db91ea6d92f65373a48c4e995a1a7aeb57009661fee675250",
"zh:68c0238cafc37433627e288fcd2c7e14f4f0afdd50b4f265d8d1f1addab6f19f",
"zh:7e6d69720734455eb1c69880f049650276089b7fa09085e130d224abaeec887a",
"zh:95bd93a696ec050c1cb5e724498fd12b1d69760d01e97c869be3252025691434",
"zh:b1b075049e33aa08c032f41a497351c9894f16287a4449032d8b805bc6dcb596",
"zh:ba91aa853372c828f808c09dbab2a5bc9493a7cf93210d1487f9637b2cac8ca4",
"zh:bc43d27dfe014266697c2ac259f4311300391aa6aa7c5d23e382fe296df938d5",
"zh:d3a04d2c76bfc1f46a117b1af7870a97353319ee8f924a37fe77861519f59525",
"zh:d3da997c05a653df6cabb912c6c05ceb6bf77219b699f04daf44fd795c81c6ed",
"zh:edd0659021b6634acf0f581d1be1985a81fcd1182e3ccb43de6eac6c43be9ab4",
"zh:f588ace57b6c35d509ecaa7136e6a8049d227b0674104a1f958359b84862d8e3",
"zh:f894ed195a3b9ebbfa1ba7c5d71be06df3a96d783ff064d22dd693ace34d638e",
"zh:fb6b0d4b111fafdcb3bb9a7dbab88e2110a6ce6324de64ecf62933ee8b651ccf",
]
}
provider "registry.terraform.io/hashicorp/random" {
version = "3.1.0"
hashes = [
"h1:BZMEPucF+pbu9gsPk0G0BHx7YP04+tKdq2MrRDF1EDM=",
"h1:EPIax4Ftp2SNdB9pUfoSjxoueDoLc/Ck3EUoeX0Dvsg=",
"h1:rKYu5ZUbXwrLG1w81k7H3nce/Ys6yAxXhWcbtk36HjY=",
"zh:2bbb3339f0643b5daa07480ef4397bd23a79963cc364cdfbb4e86354cb7725bc",
"zh:3cd456047805bf639fbf2c761b1848880ea703a054f76db51852008b11008626",
"zh:4f251b0eda5bb5e3dc26ea4400dba200018213654b69b4a5f96abee815b4f5ff",
"zh:7011332745ea061e517fe1319bd6c75054a314155cb2c1199a5b01fe1889a7e2",
"zh:738ed82858317ccc246691c8b85995bc125ac3b4143043219bd0437adc56c992",
"zh:7dbe52fac7bb21227acd7529b487511c91f4107db9cc4414f50d04ffc3cab427",
"zh:a3a9251fb15f93e4cfc1789800fc2d7414bbc18944ad4c5c98f466e6477c42bc",
"zh:a543ec1a3a8c20635cf374110bd2f87c07374cf2c50617eee2c669b3ceeeaa9f",
"zh:d9ab41d556a48bd7059f0810cf020500635bfc696c9fc3adab5ea8915c1d886b",
"zh:d9e13427a7d011dbd654e591b0337e6074eef8c3b9bb11b2e39eaaf257044fd7",
"zh:f7605bd1437752114baf601bdf6931debe6dc6bfe3006eb7e9bb9080931dca8a",
]
}
provider "registry.terraform.io/hashicorp/template" {
version = "2.2.0"
hashes = [
"h1:0wlehNaxBX7GJQnPfQwTNvvAf38Jm0Nv7ssKGMaG6Og=",
"h1:94qn780bi1qjrbC3uQtjJh3Wkfwd5+tTtJHOb7KTg9w=",
"h1:LN84cu+BZpVRvYlCzrbPfCRDaIelSyEx/W9Iwwgbnn4=",
"zh:01702196f0a0492ec07917db7aaa595843d8f171dc195f4c988d2ffca2a06386",
"zh:09aae3da826ba3d7df69efeb25d146a1de0d03e951d35019a0f80e4f58c89b53",
"zh:09ba83c0625b6fe0a954da6fbd0c355ac0b7f07f86c91a2a97849140fea49603",
"zh:0e3a6c8e16f17f19010accd0844187d524580d9fdb0731f675ffcf4afba03d16",
"zh:45f2c594b6f2f34ea663704cc72048b212fe7d16fb4cfd959365fa997228a776",
"zh:77ea3e5a0446784d77114b5e851c970a3dde1e08fa6de38210b8385d7605d451",
"zh:8a154388f3708e3df5a69122a23bdfaf760a523788a5081976b3d5616f7d30ae",
"zh:992843002f2db5a11e626b3fc23dc0c87ad3729b3b3cff08e32ffb3df97edbde",
"zh:ad906f4cebd3ec5e43d5cd6dc8f4c5c9cc3b33d2243c89c5fc18f97f7277b51d",
"zh:c979425ddb256511137ecd093e23283234da0154b7fa8b21c2687182d9aea8b2",
]
}

View File

@ -27,20 +27,10 @@
```
droplet_ip = 178.128.39.201
gateway_url = https://faasd.example.com/
```
8) View the output for sensitive data via `terraform output` command
```bash
terraform output login_cmd
login_cmd = faas-cli login -g http://178.128.39.201:8080/ -p rvIU49CEcFcHmqxj
terraform output password
login_cmd = faas-cli login -g https://faasd.example.com/ -p rvIU49CEcFcHmqxj
password = rvIU49CEcFcHmqxj
```
9) Use your browser to access the OpenFaaS interface
8) Use your browser to access the OpenFaaS interface
Note that the user-data may take a couple of minutes to come up since it will be pulling in various components and preparing the machine.
Also take into consideration the DNS propagation time for the new DNS record.

View File

@ -1,9 +1,57 @@
#! /bin/bash
#cloud-config
ssh_authorized_keys:
- ${ssh_key}
mkdir -p /var/lib/faasd/secrets/
echo ${gw_password} > /var/lib/faasd/secrets/basic-auth-password
groups:
- caddy
export FAASD_DOMAIN=${faasd_domain_name}
export LETSENCRYPT_EMAIL=${letsencrypt_email}
users:
- name: caddy
gecos: Caddy web server
primary_group: caddy
groups: caddy
shell: /usr/sbin/nologin
homedir: /var/lib/caddy
curl -sfL https://raw.githubusercontent.com/openfaas/faasd/master/hack/install.sh | bash -s -
write_files:
- content: |
{
email ${letsencrypt_email}
}
${faasd_domain_name} {
reverse_proxy 127.0.0.1:8080
}
path: /etc/caddy/Caddyfile
package_update: true
packages:
- runc
runcmd:
- curl -sLSf https://github.com/containerd/containerd/releases/download/v1.3.5/containerd-1.3.5-linux-amd64.tar.gz > /tmp/containerd.tar.gz && tar -xvf /tmp/containerd.tar.gz -C /usr/local/bin/ --strip-components=1
- curl -SLfs https://raw.githubusercontent.com/containerd/containerd/v1.3.5/containerd.service | tee /etc/systemd/system/containerd.service
- systemctl daemon-reload && systemctl start containerd
- /sbin/sysctl -w net.ipv4.conf.all.forwarding=1
- mkdir -p /opt/cni/bin
- curl -sSL https://github.com/containernetworking/plugins/releases/download/v0.8.5/cni-plugins-linux-amd64-v0.8.5.tgz | tar -xz -C /opt/cni/bin
- mkdir -p /go/src/github.com/openfaas/
- mkdir -p /var/lib/faasd/secrets/
- echo ${gw_password} > /var/lib/faasd/secrets/basic-auth-password
- echo admin > /var/lib/faasd/secrets/basic-auth-user
- cd /go/src/github.com/openfaas/ && git clone https://github.com/openfaas/faasd && cd faasd && git checkout 0.9.2
- curl -fSLs "https://github.com/openfaas/faasd/releases/download/0.9.2/faasd" --output "/usr/local/bin/faasd" && chmod a+x "/usr/local/bin/faasd"
- cd /go/src/github.com/openfaas/faasd/ && /usr/local/bin/faasd install
- systemctl status -l containerd --no-pager
- journalctl -u faasd-provider --no-pager
- systemctl status -l faasd-provider --no-pager
- systemctl status -l faasd --no-pager
- curl -sSLf https://cli.openfaas.com | sh
- sleep 5 && journalctl -u faasd --no-pager
- wget https://github.com/caddyserver/caddy/releases/download/v2.1.1/caddy_2.1.1_linux_amd64.tar.gz -O /tmp/caddy.tar.gz && tar -zxvf /tmp/caddy.tar.gz -C /usr/bin/ caddy
- wget https://raw.githubusercontent.com/caddyserver/dist/master/init/caddy.service -O /etc/systemd/system/caddy.service
- systemctl daemon-reload
- systemctl enable caddy
- systemctl start caddy

View File

@ -1,11 +1,5 @@
terraform {
required_version = ">= 1.0.4"
required_providers {
digitalocean = {
source = "digitalocean/digitalocean"
version = "2.11.0"
}
}
required_version = ">= 0.12"
}
variable "do_token" {
@ -16,7 +10,7 @@ variable "do_domain" {
}
variable "do_subdomain" {
description = "Your public subdomain"
default = "faasd"
default = "faasd"
}
variable "letsencrypt_email" {
description = "Email used to order a certificate from Letsencrypt"
@ -38,44 +32,41 @@ provider "digitalocean" {
token = var.do_token
}
data "local_file" "ssh_key"{
filename = pathexpand(var.ssh_key_file)
}
resource "random_password" "password" {
length = 16
special = true
length = 16
special = true
override_special = "_-#"
}
data "template_file" "cloud_init" {
template = file("cloud-config.tpl")
vars = {
gw_password = random_password.password.result,
faasd_domain_name = "${var.do_subdomain}.${var.do_domain}"
letsencrypt_email = var.letsencrypt_email
}
}
resource "digitalocean_ssh_key" "faasd_ssh_key" {
name = "ssh-key"
public_key = file(var.ssh_key_file)
template = "${file("cloud-config.tpl")}"
vars = {
gw_password=random_password.password.result,
ssh_key=data.local_file.ssh_key.content,
faasd_domain_name="${var.do_subdomain}.${var.do_domain}"
letsencrypt_email=var.letsencrypt_email
}
}
resource "digitalocean_droplet" "faasd" {
region = var.do_region
image = "ubuntu-18-04-x64"
name = "faasd"
size = "s-1vcpu-1gb"
region = var.do_region
image = "ubuntu-18-04-x64"
name = "faasd"
size = "s-1vcpu-1gb"
user_data = data.template_file.cloud_init.rendered
ssh_keys = [
digitalocean_ssh_key.faasd_ssh_key.id
]
}
resource "digitalocean_record" "faasd" {
domain = var.do_domain
type = "A"
name = var.do_subdomain
name = "faasd"
value = digitalocean_droplet.faasd.ipv4_address
# Only creates record if do_create_record is true
count = var.do_create_record == true ? 1 : 0
count = var.do_create_record == true ? 1 : 0
}
output "droplet_ip" {
@ -87,11 +78,9 @@ output "gateway_url" {
}
output "password" {
value = random_password.password.result
sensitive = true
value = random_password.password.result
}
output "login_cmd" {
value = "faas-cli login -g https://${var.do_subdomain}.${var.do_domain}/ -p ${random_password.password.result}"
sensitive = true
value = "faas-cli login -g https://${var.do_subdomain}.${var.do_domain}/ -p ${random_password.password.result}"
}

56
docs/bootstrap/main.tf Normal file
View File

@ -0,0 +1,56 @@
terraform {
required_version = ">= 0.12"
}
variable "do_token" {}
variable "ssh_key_file" {
default = "~/.ssh/id_rsa.pub"
description = "Path to the SSH public key file"
}
provider "digitalocean" {
token = var.do_token
}
resource "random_password" "password" {
length = 16
special = true
override_special = "_-#"
}
data "local_file" "ssh_key"{
filename = pathexpand(var.ssh_key_file)
}
data "template_file" "cloud_init" {
template = "${file("cloud-config.tpl")}"
vars = {
gw_password=random_password.password.result,
ssh_key=data.local_file.ssh_key.content,
}
}
resource "digitalocean_droplet" "faasd" {
region = "lon1"
image = "ubuntu-18-04-x64"
name = "faasd"
# Plans: https://developers.digitalocean.com/documentation/changelog/api-v2/new-size-slugs-for-droplet-plan-changes/
#size = "512mb"
size = "s-1vcpu-1gb"
user_data = data.template_file.cloud_init.rendered
}
output "password" {
value = random_password.password.result
}
output "gateway_url" {
value = "http://${digitalocean_droplet.faasd.ipv4_address}:8080/"
}
output "login_cmd" {
value = "faas-cli login -g http://${digitalocean_droplet.faasd.ipv4_address}:8080/ -p ${random_password.password.result}"
}

File diff suppressed because one or more lines are too long

Binary file not shown.

Before

Width:  |  Height:  |  Size: 48 KiB

101
go.mod
View File

@ -1,101 +0,0 @@
module github.com/openfaas/faasd
go 1.22
require (
github.com/alexellis/arkade v0.0.0-20240320084407-6cf4a641c415
github.com/compose-spec/compose-go v0.0.0-20200528042322-36d8ce368e05
github.com/containerd/containerd v1.7.22
github.com/containerd/go-cni v1.1.9
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf
github.com/docker/cli v24.0.7+incompatible
github.com/docker/distribution v2.8.3+incompatible
github.com/docker/docker v24.0.7+incompatible // indirect
github.com/docker/go-units v0.5.0
github.com/gorilla/mux v1.8.1
github.com/morikuni/aec v1.0.0
github.com/opencontainers/runtime-spec v1.1.0
github.com/openfaas/faas-provider v0.25.3
github.com/pkg/errors v0.9.1
github.com/sethvargo/go-password v0.2.0
github.com/spf13/cobra v1.8.0
github.com/spf13/pflag v1.0.5
github.com/vishvananda/netlink v1.2.1-beta.2
github.com/vishvananda/netns v0.0.4
golang.org/x/sys v0.19.0
k8s.io/apimachinery v0.29.3
)
require (
github.com/alexellis/go-execute/v2 v2.2.1
github.com/distribution/reference v0.6.0
)
require (
github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24 // indirect
github.com/AdamKorcz/go-118-fuzz-build v0.0.0-20230306123547-8075edf89bb0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/Microsoft/hcsshim v0.11.7 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/containerd/cgroups v1.1.0 // indirect
github.com/containerd/containerd/api v1.7.19 // indirect
github.com/containerd/continuity v0.4.2 // indirect
github.com/containerd/errdefs v0.1.0 // indirect
github.com/containerd/fifo v1.1.0 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/platforms v0.2.1 // indirect
github.com/containerd/ttrpc v1.2.5 // indirect
github.com/containerd/typeurl/v2 v2.1.1 // indirect
github.com/containernetworking/cni v1.1.2 // indirect
github.com/docker/docker-credential-helpers v0.8.0 // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
github.com/felixge/httpsnoop v1.0.3 // indirect
github.com/go-logr/logr v1.3.0 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/uuid v1.4.0 // indirect
github.com/imdario/mergo v0.3.14 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/klauspost/compress v1.17.4 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/mattn/go-shellwords v1.0.12 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/moby/locker v1.0.1 // indirect
github.com/moby/sys/mountinfo v0.6.2 // indirect
github.com/moby/sys/sequential v0.5.0 // indirect
github.com/moby/sys/signal v0.7.0 // indirect
github.com/moby/sys/user v0.3.0 // indirect
github.com/moby/sys/userns v0.1.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0 // indirect
github.com/opencontainers/selinux v1.11.0 // indirect
github.com/prometheus/client_golang v1.19.0 // indirect
github.com/prometheus/client_model v0.6.0 // indirect
github.com/prometheus/common v0.51.1 // indirect
github.com/prometheus/procfs v0.13.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xeipuuv/gojsonschema v1.2.0 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.45.0 // indirect
go.opentelemetry.io/otel v1.21.0 // indirect
go.opentelemetry.io/otel/metric v1.21.0 // indirect
go.opentelemetry.io/otel/trace v1.21.0 // indirect
golang.org/x/net v0.24.0 // indirect
golang.org/x/sync v0.6.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/tools v0.19.0 // indirect
google.golang.org/genproto v0.0.0-20231211222908-989df2bf70f3 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20231212172506-995d672761c0 // indirect
google.golang.org/grpc v1.59.0 // indirect
google.golang.org/protobuf v1.33.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gotest.tools/v3 v3.0.3 // indirect
)

405
go.sum
View File

@ -1,405 +0,0 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24 h1:bvDV9vkmnHYOMsOr4WLk+Vo07yKIzd94sVoIqshQ4bU=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8=
github.com/AdamKorcz/go-118-fuzz-build v0.0.0-20230306123547-8075edf89bb0 h1:59MxjQVfjXsBpLy+dbd2/ELV5ofnUkUZBvWSC85sheA=
github.com/AdamKorcz/go-118-fuzz-build v0.0.0-20230306123547-8075edf89bb0/go.mod h1:OahwfttHWG6eJ0clwcfBAHoDI6X/LV/15hx/wlMZSrU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/Microsoft/hcsshim v0.11.5 h1:haEcLNpj9Ka1gd3B3tAEs9CpE0c+1IhoL59w/exYU38=
github.com/Microsoft/hcsshim v0.11.5/go.mod h1:MV8xMfmECjl5HdO7U/3/hFVnkmSBjAjmA09d4bExKcU=
github.com/Microsoft/hcsshim v0.11.7 h1:vl/nj3Bar/CvJSYo7gIQPyRWc9f3c6IeSNavBTSZNZQ=
github.com/Microsoft/hcsshim v0.11.7/go.mod h1:MV8xMfmECjl5HdO7U/3/hFVnkmSBjAjmA09d4bExKcU=
github.com/alexellis/arkade v0.0.0-20240320084407-6cf4a641c415 h1:mLD1eSfXbmXcwKMP1AsFl01G2U16aC9E22Tcjehyyrw=
github.com/alexellis/arkade v0.0.0-20240320084407-6cf4a641c415/go.mod h1:3sT9Gq9WUFG9Wwz9dFbRRc/2L3yepsA4p272aG2DR6w=
github.com/alexellis/go-execute/v2 v2.2.1 h1:4Ye3jiCKQarstODOEmqDSRCqxMHLkC92Bhse743RdOI=
github.com/alexellis/go-execute/v2 v2.2.1/go.mod h1:FMdRnUTiFAmYXcv23txrp3VYZfLo24nMpiIneWgKHTQ=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/compose-spec/compose-go v0.0.0-20200528042322-36d8ce368e05 h1:3CoRXflT4IAdIpsFTqZBlLuHOkHYhBvW0iijkQKm4QY=
github.com/compose-spec/compose-go v0.0.0-20200528042322-36d8ce368e05/go.mod h1:y75QUr1jcR5aFNf3Tj3dhwnujABGz6UaRrZ5qZwF1cc=
github.com/containerd/cgroups v1.1.0 h1:v8rEWFl6EoqHB+swVNjVoCJE8o3jX7e8nqBGPLaDFBM=
github.com/containerd/cgroups v1.1.0/go.mod h1:6ppBcbh/NOOUU+dMKrykgaBnK9lCIBxHqJDGwsa1mIw=
github.com/containerd/containerd v1.7.18 h1:jqjZTQNfXGoEaZdW1WwPU0RqSn1Bm2Ay/KJPUuO8nao=
github.com/containerd/containerd v1.7.18/go.mod h1:IYEk9/IO6wAPUz2bCMVUbsfXjzw5UNP5fLz4PsUygQ4=
github.com/containerd/containerd v1.7.22 h1:nZuNnNRA6T6jB975rx2RRNqqH2k6ELYKDZfqTHqwyy0=
github.com/containerd/containerd v1.7.22/go.mod h1:e3Jz1rYRUZ2Lt51YrH9Rz0zPyJBOlSvB3ghr2jbVD8g=
github.com/containerd/containerd/api v1.7.19 h1:VWbJL+8Ap4Ju2mx9c9qS1uFSB1OVYr5JJrW2yT5vFoA=
github.com/containerd/containerd/api v1.7.19/go.mod h1:fwGavl3LNwAV5ilJ0sbrABL44AQxmNjDRcwheXDb6Ig=
github.com/containerd/continuity v0.4.2 h1:v3y/4Yz5jwnvqPKJJ+7Wf93fyWoCB3F5EclWG023MDM=
github.com/containerd/continuity v0.4.2/go.mod h1:F6PTNCKepoxEaXLQp3wDAjygEnImnZ/7o4JzpodfroQ=
github.com/containerd/errdefs v0.1.0 h1:m0wCRBiu1WJT/Fr+iOoQHMQS/eP5myQ8lCv4Dz5ZURM=
github.com/containerd/errdefs v0.1.0/go.mod h1:YgWiiHtLmSeBrvpw+UfPijzbLaB77mEG1WwJTDETIV0=
github.com/containerd/fifo v1.1.0 h1:4I2mbh5stb1u6ycIABlBw9zgtlK8viPI9QkQNRQEEmY=
github.com/containerd/fifo v1.1.0/go.mod h1:bmC4NWMbXlt2EZ0Hc7Fx7QzTFxgPID13eH0Qu+MAb2o=
github.com/containerd/go-cni v1.1.9 h1:ORi7P1dYzCwVM6XPN4n3CbkuOx/NZ2DOqy+SHRdo9rU=
github.com/containerd/go-cni v1.1.9/go.mod h1:XYrZJ1d5W6E2VOvjffL3IZq0Dz6bsVlERHbekNK90PM=
github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I=
github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo=
github.com/containerd/platforms v0.2.1 h1:zvwtM3rz2YHPQsF2CHYM8+KtB5dvhISiXh5ZpSBQv6A=
github.com/containerd/platforms v0.2.1/go.mod h1:XHCb+2/hzowdiut9rkudds9bE5yJ7npe7dG/wG+uFPw=
github.com/containerd/ttrpc v1.2.4 h1:eQCQK4h9dxDmpOb9QOOMh2NHTfzroH1IkmHiKZi05Oo=
github.com/containerd/ttrpc v1.2.4/go.mod h1:ojvb8SJBSch0XkqNO0L0YX/5NxR3UnVk2LzFKBK0upc=
github.com/containerd/ttrpc v1.2.5 h1:IFckT1EFQoFBMG4c3sMdT8EP3/aKfumK1msY+Ze4oLU=
github.com/containerd/ttrpc v1.2.5/go.mod h1:YCXHsb32f+Sq5/72xHubdiJRQY9inL4a4ZQrAbN1q9o=
github.com/containerd/typeurl/v2 v2.1.1 h1:3Q4Pt7i8nYwy2KmQWIw2+1hTvwTE/6w9FqcttATPO/4=
github.com/containerd/typeurl/v2 v2.1.1/go.mod h1:IDp2JFvbwZ31H8dQbEIY7sDl2L3o3HZj1hsSQlywkQ0=
github.com/containernetworking/cni v1.1.2 h1:wtRGZVv7olUHMOqouPpn3cXJWpJgM6+EUl31EQbXALQ=
github.com/containernetworking/cni v1.1.2/go.mod h1:sDpYKmGVENF3s6uvMvGgldDWeG8dMxakj/u+i9ht9vw=
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf h1:iW4rZ826su+pqaw19uhpSCzhj44qo35pNgKFGqzDKkU=
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/distribution/reference v0.5.0 h1:/FUIFXtfc/x2gpa5/VGfiGLuOIdYa1t65IKK2OFGvA0=
github.com/distribution/reference v0.5.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/docker/cli v24.0.7+incompatible h1:wa/nIwYFW7BVTGa7SWPVyyXU9lgORqUb1xfI36MSkFg=
github.com/docker/cli v24.0.7+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk=
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v24.0.7+incompatible h1:Wo6l37AuwP3JaMnZa226lzVXGA3F9Ig1seQen0cKYlM=
github.com/docker/docker v24.0.7+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.8.0 h1:YQFtbBQb4VrpoPxhFuzEBPQ9E16qz5SpHLS+uswaCp8=
github.com/docker/docker-credential-helpers v0.8.0/go.mod h1:UGFXcuoQ5TxPiB54nHOZ32AWRqQdECoh/Mg0AlEYb40=
github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c h1:+pKlWGMw7gf6bQ+oDZB4KHQFypsfjYlq/C4rfL7D3g8=
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c/go.mod h1:Uw6UezgYA44ePAFQYUehOuCzmy5zmg/+nl2ZfMWGkpA=
github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/felixge/httpsnoop v1.0.3 h1:s/nj+GCswXYzN5v2DpNMuMQYe+0DDwt5WVCU6CWBdXk=
github.com/felixge/httpsnoop v1.0.3/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.3.0 h1:2y3SDp0ZXuc6/cjLSZ+Q3ir+QB9T/iG5yYRXqsagWSY=
github.com/go-logr/logr v1.3.0/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 h1:K6RDEckDVWvDI9JAJYCmNdQXq6neHJOYx3V6jnqNEec=
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.3.1 h1:KjJaJ9iWZ3jOFZIf1Lqf4laDRCasjl0BCmnEGxkdLb4=
github.com/google/uuid v1.3.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.4.0 h1:MtMxsa51/r9yyhkyLsVeVt0B+BGQZzpQiTQ4eHZ8bc4=
github.com/google/uuid v1.4.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/imdario/mergo v0.3.8/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/imdario/mergo v0.3.14 h1:fOqeC1+nCuuk6PKQdg9YmosXX7Y7mHX6R/0ZldI9iHo=
github.com/imdario/mergo v0.3.14/go.mod h1:WBLT9ZmE3lPoWsEzCh9LPo3TiwVN+ZKEjmz+hD27ysY=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.17.4 h1:Ej5ixsIri7BrIjBkRZLTo6ghwrEtHFk7ijlczPW4fZ4=
github.com/klauspost/compress v1.17.4/go.mod h1:/dCuZOvVtNoHsyb+cuJD3itjs3NbnF6KH9zAO4BDxPM=
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mattn/go-shellwords v1.0.10/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
github.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk=
github.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
github.com/mitchellh/mapstructure v1.3.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/moby/locker v1.0.1 h1:fOXqR41zeveg4fFODix+1Ch4mj/gT0NE1XJbp/epuBg=
github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc=
github.com/moby/sys/mountinfo v0.6.2 h1:BzJjoreD5BMFNmD9Rus6gdd1pLuecOFPt8wC+Vygl78=
github.com/moby/sys/mountinfo v0.6.2/go.mod h1:IJb6JQeOklcdMU9F5xQ8ZALD+CUr5VlGpwtX+VE0rpI=
github.com/moby/sys/sequential v0.5.0 h1:OPvI35Lzn9K04PBbCLW0g4LcFAJgHsvXsRyewg5lXtc=
github.com/moby/sys/sequential v0.5.0/go.mod h1:tH2cOOs5V9MlPiXcQzRC+eEyab644PWKGRYaaV5ZZlo=
github.com/moby/sys/signal v0.7.0 h1:25RW3d5TnQEoKvRbEKUGay6DCQ46IxAVTT9CUMgmsSI=
github.com/moby/sys/signal v0.7.0/go.mod h1:GQ6ObYZfqacOwTtlXvcmh9A26dVRul/hbOZn88Kg8Tg=
github.com/moby/sys/user v0.1.0 h1:WmZ93f5Ux6het5iituh9x2zAG7NFY9Aqi49jjE1PaQg=
github.com/moby/sys/user v0.1.0/go.mod h1:fKJhFOnsCN6xZ5gSfbM6zaHGgDJMrqt9/reuj4T7MmU=
github.com/moby/sys/user v0.3.0 h1:9ni5DlcW5an3SvRSx4MouotOygvzaXbaSrc/wGDFWPo=
github.com/moby/sys/user v0.3.0/go.mod h1:bG+tYYYJgaMtRKgEmuueC0hJEAZWwtIbZTB+85uoHjs=
github.com/moby/sys/userns v0.1.0 h1:tVLXkFOxVu9A64/yh59slHVv9ahO9UIev4JZusOLG/g=
github.com/moby/sys/userns v0.1.0/go.mod h1:IHUYgu/kao6N8YZlp9Cf444ySSvCmDlmzUcYfDHOl28=
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=
github.com/onsi/ginkgo v1.16.4 h1:29JGrr5oVBm5ulCWet69zQkzWipVXIol6ygQUe/EzNc=
github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0=
github.com/onsi/ginkgo/v2 v2.1.3/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c=
github.com/onsi/ginkgo/v2 v2.13.0 h1:0jY9lJquiL8fcf3M4LAXN5aMlS/b2BV86HFFPCPMgE4=
github.com/onsi/ginkgo/v2 v2.13.0/go.mod h1:TE309ZR8s5FsKKpuB1YAQYBzCaAfUgatB/xlT/ETL/o=
github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=
github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY=
github.com/onsi/gomega v1.29.0 h1:KIA/t2t5UBzoirT4H9tsML45GEbo3ouUnBHsCfD2tVg=
github.com/onsi/gomega v1.29.0/go.mod h1:9sxs+SwGrKI0+PWe4Fxa9tFQQBG5xSsSbMXOI8PPpoQ=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM=
github.com/opencontainers/runtime-spec v1.1.0 h1:HHUyrt9mwHUjtasSbXSMvs4cyFxh+Bll4AjJ9odEGpg=
github.com/opencontainers/runtime-spec v1.1.0/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/selinux v1.11.0 h1:+5Zbo97w3Lbmb3PeqQtpmTkMwsW5nRI3YaLpt7tQ7oU=
github.com/opencontainers/selinux v1.11.0/go.mod h1:E5dMC3VPuVvVHDYmi78qvhJp8+M586T4DlDRYpFkyec=
github.com/openfaas/faas-provider v0.25.3 h1:cy5GKP1R/xZkPjg+9We7yqpfz298GrKw4ZRYJVprt7Q=
github.com/openfaas/faas-provider v0.25.3/go.mod h1:NsETIfEndZn4mn/w/XnBTcDTwKqULCziphLp7KgeRcA=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.19.0 h1:ygXvpU1AoN1MhdzckN+PyD9QJOSD4x7kmXYlnfbA6JU=
github.com/prometheus/client_golang v1.19.0/go.mod h1:ZRM9uEAypZakd+q/x7+gmsvXdURP+DABIEIjnmDdp+k=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.6.0 h1:k1v3CzpSRUTrKMppY35TLwPvxHqBu0bYgxZzqGIgaos=
github.com/prometheus/client_model v0.6.0/go.mod h1:NTQHnmxFpouOD0DpvP4XujX3CdOAGQPoaGhyTchlyt8=
github.com/prometheus/common v0.51.1 h1:eIjN50Bwglz6a/c3hAgSMcofL3nD+nFQkV6Dd4DsQCw=
github.com/prometheus/common v0.51.1/go.mod h1:lrWtQx+iDfn2mbH5GUzlH9TSHyfZpHkSiG1W7y3sF2Q=
github.com/prometheus/procfs v0.13.0 h1:GqzLlQyfsPbaEHaQkO7tbDlriv/4o5Hudv6OXHGKX7o=
github.com/prometheus/procfs v0.13.0/go.mod h1:cd4PFCR54QLnGKPaKGA6l+cfuNXtht43ZKY6tow0Y1g=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sethvargo/go-password v0.2.0 h1:BTDl4CC/gjf/axHMaDQtw507ogrXLci6XRiLc7i/UHI=
github.com/sethvargo/go-password v0.2.0/go.mod h1:Ym4Mr9JXLBycr02MFuVQ/0JHidNetSgbzutTr3zsYXE=
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0 h1:1zr/of2m5FGMsad5YfcqgdqdWrIhu+EBEJRhR1U7z/c=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/vishvananda/netlink v1.2.1-beta.2 h1:Llsql0lnQEbHj0I1OuKyp8otXp0r3q0mPkuhwHfStVs=
github.com/vishvananda/netlink v1.2.1-beta.2/go.mod h1:twkDnbuQxJYemMlGd4JFIcuhgX83tXhKS2B/PRMpOho=
github.com/vishvananda/netns v0.0.0-20200728191858-db3c7e526aae/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0=
github.com/vishvananda/netns v0.0.4 h1:Oeaw1EM2JMxD51g9uhtC0D7erkIjgmj8+JZc26m1YX8=
github.com/vishvananda/netns v0.0.4/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb h1:zGWFAtiMcyryUHoUjUJX0/lt1H2+i2Ka2n+D3DImSNo=
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 h1:EzJWgHovont7NscjpAxXsDA8S8BMYve8Y5+7cuRE7R0=
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ=
github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17UxZ74=
github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.45.0 h1:x8Z78aZx8cOF0+Kkazoc7lwUNMGy0LrzEMxTm4BbTxg=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.45.0/go.mod h1:62CPTSry9QZtOaSsE3tOzhx6LzDhHnXJ6xHeMNNiM6Q=
go.opentelemetry.io/otel v1.19.0 h1:MuS/TNf4/j4IXsZuJegVzI1cwut7Qc00344rgH7p8bs=
go.opentelemetry.io/otel v1.19.0/go.mod h1:i0QyjOq3UPoTzff0PJB2N66fb4S0+rSbSB15/oyH9fY=
go.opentelemetry.io/otel v1.21.0 h1:hzLeKBZEL7Okw2mGzZ0cc4k/A7Fta0uoPgaJCr8fsFc=
go.opentelemetry.io/otel v1.21.0/go.mod h1:QZzNPQPm1zLX4gZK4cMi+71eaorMSGT3A4znnUvNNEo=
go.opentelemetry.io/otel/metric v1.19.0 h1:aTzpGtV0ar9wlV4Sna9sdJyII5jTVJEvKETPiOKwvpE=
go.opentelemetry.io/otel/metric v1.19.0/go.mod h1:L5rUsV9kM1IxCj1MmSdS+JQAcVm319EUrDVLrt7jqt8=
go.opentelemetry.io/otel/metric v1.21.0 h1:tlYWfeo+Bocx5kLEloTjbcDwBuELRrIFxwdQ36PlJu4=
go.opentelemetry.io/otel/metric v1.21.0/go.mod h1:o1p3CA8nNHW8j5yuQLdc1eeqEaPfzug24uvsyIEJRWM=
go.opentelemetry.io/otel/trace v1.19.0 h1:DFVQmlVbfVeOuBRrwdtaehRrWiL1JoVs9CPIQ1Dzxpg=
go.opentelemetry.io/otel/trace v1.19.0/go.mod h1:mfaSyvGyEJEI0nyV2I4qhNQnbBOUUmYZpYojqMnX2vo=
go.opentelemetry.io/otel/trace v1.21.0 h1:WD9i5gzvoUPuXIXH24ZNBudiarZDKuekPqi/E8fpfLc=
go.opentelemetry.io/otel/trace v1.21.0/go.mod h1:LGbsEB0f9LGjN+OZaQQ26sohbOmiMR+BaslueVtS/qQ=
go.uber.org/goleak v1.2.1 h1:NBol2c7O1ZokfZ0LEU9K6Whx/KnwvepVetCUhtKja4A=
go.uber.org/goleak v1.2.1/go.mod h1:qlT2yGI9QafXHhZZLxlSuNsMw3FFLxBr+tBRlmO1xH4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=
golang.org/x/net v0.24.0 h1:1PcaxkF854Fu3+lvBIx5SYn9wRlBzzcnHZSiaFFAb0w=
golang.org/x/net v0.24.0/go.mod h1:2Q7sJY5mzlzWjKtYUEXSlBWCdyaioyXzRB2RtU8KVE8=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200217220822-9197077df867/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200728102440-3e129f6d46b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.19.0 h1:q5f1RH2jigJ1MoAWp2KTp3gm5zAGFUTarQZ5U386+4o=
golang.org/x/sys v0.19.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190624222133-a101b041ded4/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.19.0 h1:tfGCXNR1OsFG+sVdLAitlpjAvD/I6dHDKnYrpEZUHkw=
golang.org/x/tools v0.19.0/go.mod h1:qoJWxmGSIBmAeriMx19ogtrEPrGtDbPK634QFIcLAhc=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20230920204549-e6e6cdab5c13 h1:vlzZttNJGVqTsRFU9AmdnrcO1Znh8Ew9kCD//yjigk0=
google.golang.org/genproto v0.0.0-20230920204549-e6e6cdab5c13/go.mod h1:CCviP9RmpZ1mxVr8MUjCnSiY09IbAXZxhLE6EhHIdPU=
google.golang.org/genproto v0.0.0-20231211222908-989df2bf70f3 h1:1hfbdAfFbkmpg41000wDVqr7jUpK/Yo+LPnIxxGzmkg=
google.golang.org/genproto v0.0.0-20231211222908-989df2bf70f3/go.mod h1:5RBcpGRxr25RbDzY5w+dmaqpSEvl8Gwl1x2CICf60ic=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231002182017-d307bd883b97 h1:6GQBEOdGkX6MMTLT9V+TjtIRZCw9VPD5Z+yHY9wMgS0=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231002182017-d307bd883b97/go.mod h1:v7nGkzlmW8P3n/bKmWBn2WpBjpOEx8Q6gMueudAmKfY=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231212172506-995d672761c0 h1:/jFB8jK5R3Sq3i/lmeZO0cATSzFfZaJq1J2Euan3XKU=
google.golang.org/genproto/googleapis/rpc v0.0.0-20231212172506-995d672761c0/go.mod h1:FUoWkonphQm3RhTS+kOEhF8h0iDpm4tdXolVCeZ9KKA=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/grpc v1.59.0 h1:Z5Iec2pjwb+LEOqzpB2MR12/eKFhDPhuqW91O+4bwUk=
google.golang.org/grpc v1.59.0/go.mod h1:aUPDwccQo6OTjy7Hct4AfBPD1GptF4fyUjIkQ9YtF98=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk=
gotest.tools/v3 v3.0.3 h1:4AuOwCGf4lLR9u3YOe2awrHygurzhO/HeQ6laiA6Sx0=
gotest.tools/v3 v3.0.3/go.mod h1:Z7Lb0S5l+klDB31fvDQX8ss/FlKDxtlFlw3Oa8Ymbl8=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
k8s.io/apimachinery v0.29.3 h1:2tbx+5L7RNvqJjn7RIuIKu9XTsIZ9Z5wX2G22XAa5EU=
k8s.io/apimachinery v0.29.3/go.mod h1:hx/S4V2PNW4OMg3WizRrHutyB5la0iCUbZym+W0EQIU=

View File

@ -1,16 +1,12 @@
#!/bin/bash
# See pre-reqs:
# https://github.com/alexellis/containerd-arm
export ARCH="armv6l"
echo "Downloading Go"
export ARCH="arm64"
if [ ! -d "/usr/local/go/bin" ]; then
curl -sLS https://get.arkade.dev | sudo sh
sudo -E arkade system install go
else
echo "Go already present, skipping."
fi
curl -SLsf https://dl.google.com/go/go1.12.14.linux-$ARCH.tar.gz --output /tmp/go.tgz
sudo rm -rf /usr/local/go/
sudo mkdir -p /usr/local/go/
sudo tar -xvf /tmp/go.tgz -C /usr/local/go/ --strip-components=1
export GOPATH=$HOME/go/
export PATH=$PATH:/usr/local/go/bin/
@ -25,7 +21,7 @@ git clone https://github.com/containerd/containerd
cd containerd
git fetch origin --tags
git checkout v1.7.22
git checkout v1.3.2
make
sudo make install

View File

@ -1,17 +1,12 @@
#!/bin/bash
export ARCH="amd64"
echo "Downloading Go"
# See pre-reqs:
# https://github.com/alexellis/containerd-arm
export ARCH="arm64"
if [ ! -d "/usr/local/go/bin" ]; then
curl -sLS https://get.arkade.dev | sudo sh
sudo -E arkade system install go
else
echo "Go already present, skipping."
fi
curl -SLsf https://dl.google.com/go/go1.12.14.linux-$ARCH.tar.gz --output /tmp/go.tgz
sudo rm -rf /usr/local/go/
sudo mkdir -p /usr/local/go/
sudo tar -xvf /tmp/go.tgz -C /usr/local/go/ --strip-components=1
export GOPATH=$HOME/go/
export PATH=$PATH:/usr/local/go/bin/
@ -26,7 +21,7 @@ git clone https://github.com/containerd/containerd
cd containerd
git fetch origin --tags
git checkout v1.7.22
git checkout v1.3.2
make
sudo make install

View File

@ -2,10 +2,9 @@
Description=faasd-provider
[Service]
MemoryMax=500M
MemoryLimit=500M
Environment="secret_mount_path={{.SecretMountPath}}"
Environment="basic_auth=true"
Environment="hosts_dir=/var/lib/faasd"
ExecStart=/usr/local/bin/faasd provider
Restart=on-failure
RestartSec=10s

View File

@ -3,7 +3,7 @@ Description=faasd
After=faasd-provider.service
[Service]
MemoryMax=500M
MemoryLimit=500M
ExecStart=/usr/local/bin/faasd up
Restart=on-failure
RestartSec=10s

View File

@ -1,99 +0,0 @@
#!/bin/bash
set -e # stop on error
set -o pipefail
if [ "$EUID" -ne 0 ]; then
echo "Please run as root or with sudo"
exit
fi
has_yum() {
[ -n "$(command -v yum)" ]
}
has_apt_get() {
[ -n "$(command -v apt-get)" ]
}
has_pacman() {
[ -n "$(command -v pacman)" ]
}
install_required_packages() {
if $(has_apt_get); then
# Debian bullseye is missing iptables. Added to required packages
# to get it working in raspberry pi. No such known issues in
# other distros. Hence, adding only to this block.
# reference: https://github.com/openfaas/faasd/pull/237
apt-get update -y
apt-get install -y curl runc bridge-utils iptables
elif $(has_yum); then
yum check-update -y
yum install -y curl runc iptables-services which
elif $(has_pacman); then
pacman -Syy
pacman -Sy curl runc bridge-utils
else
fatal "Could not find apt-get, yum, or pacman. Cannot install dependencies on this OS."
exit 1
fi
}
echo "OpenFaaS Edge (based upon faasd and OpenFaaS Standard)"
echo ""
echo ""
echo "1. Installing required OS packages, set SKIP_OS=1 to skip this step"
echo ""
if [ -z "$SKIP_OS" ]; then
install_required_packages
fi
echo "2. Downloading OCI image, and installing pre-requisites"
echo ""
if [ ! -x "$(command -v arkade)" ]; then
# For Centos, RHEL, Fedora, Amazon Linux, and Oracle Linux, use BINLOCATION=/usr/bin/
if $(has_yum); then
BINLOCATION=/usr/bin/
fi
curl -sLS https://get.arkade.dev | BINLOCATION=${BINLOCATION} sh
fi
PATH=$PATH:$HOME/.arkade/bin
tmpdir=$(mktemp -d)
# Ensure all existing services are stopped when installing over an
# existing faasd installation
systemctl stop faasd || :
systemctl stop faasd-provider || :
systemctl stop containerd || :
killall -9 containerd-shim-runc-v2 || :
killall -9 faasd || :
# crane, or docker can also be used to download the OCI image and to extract it
# Rather than the :latest tag, a specific tag can be given
# Use "crane ls ghcr.io/openfaasltd/faasd-pro" to see available tags
${BINLOCATION}arkade oci install --path ${tmpdir} \
ghcr.io/openfaasltd/faasd-pro:latest
cd ${tmpdir}
./install.sh ./
echo ""
echo "3. You now need to activate your license via GitHub"
echo ""
echo "sudo -E faasd github login"
echo "sudo -E faasd activate"
echo ""
echo ""
echo "4. Then perform the final installation steps"
echo ""
echo "sudo -E sh -c \"cd ${tmpdir}/var/lib/faasd && faasd install\""
echo ""

View File

@ -1,186 +0,0 @@
#!/bin/bash
# Copyright OpenFaaS Author(s) 2022
set -e -x -o pipefail
export OWNER="openfaas"
export REPO="faasd"
# On CentOS /usr/local/bin is not included in the PATH when using sudo.
# Running arkade with sudo on CentOS requires the full path
# to the arkade binary.
export ARKADE=/usr/local/bin/arkade
# When running as a startup script (cloud-init), the HOME variable is not always set.
# As it is required for arkade to properly download tools,
# set the variable to /usr/local so arkade will download binaries to /usr/local/.arkade
if [ -z "${HOME}" ]; then
export HOME=/usr/local
fi
version=""
echo "Finding latest version from GitHub"
version=$(curl -sI https://github.com/$OWNER/$REPO/releases/latest | grep -i "location:" | awk -F"/" '{ printf "%s", $NF }' | tr -d '\r')
echo "$version"
if [ ! $version ]; then
echo "Failed while attempting to get latest version"
exit 1
fi
SUDO=sudo
if [ "$(id -u)" -eq 0 ]; then
SUDO=
fi
verify_system() {
arch=$(uname -m)
if [ "$arch" == "armv7l" ]; then
fatal 'faasd requires a 64-bit Operating System, see: https://github.com/openfaas/faasd/issues/364'
fi
if ! [ -d /run/systemd ]; then
fatal 'Can not find systemd to use as a process supervisor for faasd'
fi
}
has_yum() {
[ -n "$(command -v yum)" ]
}
has_apt_get() {
[ -n "$(command -v apt-get)" ]
}
has_pacman() {
[ -n "$(command -v pacman)" ]
}
install_required_packages() {
if $(has_apt_get); then
# Debian bullseye is missing iptables. Added to required packages
# to get it working in raspberry pi. No such known issues in
# other distros. Hence, adding only to this block.
# reference: https://github.com/openfaas/faasd/pull/237
$SUDO apt-get update -y
$SUDO apt-get install -y curl runc bridge-utils iptables
elif $(has_yum); then
$SUDO yum check-update -y
$SUDO yum install -y curl runc iptables-services
elif $(has_pacman); then
$SUDO pacman -Syy
$SUDO pacman -Sy curl runc bridge-utils
else
fatal "Could not find apt-get, yum, or pacman. Cannot install dependencies on this OS."
exit 1
fi
}
install_arkade(){
curl -sLS https://get.arkade.dev | $SUDO sh
arkade --help
}
install_cni_plugins() {
cni_version=v0.9.1
$SUDO $ARKADE system install cni --version ${cni_version} --path /opt/cni/bin --progress=false
}
install_containerd() {
CONTAINERD_VER=1.7.22
$SUDO systemctl unmask containerd || :
arch=$(uname -m)
$SUDO $ARKADE system install containerd --systemd --version v${CONTAINERD_VER} --progress=false
sleep 5
}
install_faasd() {
arch=$(uname -m)
case $arch in
x86_64 | amd64)
suffix=""
;;
aarch64)
suffix=-arm64
;;
*)
echo "Unsupported architecture $arch"
exit 1
;;
esac
$SUDO curl -fSLs "https://github.com/openfaas/faasd/releases/download/${version}/faasd${suffix}" --output "/usr/local/bin/faasd"
$SUDO chmod a+x "/usr/local/bin/faasd"
mkdir -p /tmp/faasd-${version}-installation/hack
cd /tmp/faasd-${version}-installation
$SUDO curl -fSLs "https://raw.githubusercontent.com/openfaas/faasd/${version}/docker-compose.yaml" --output "docker-compose.yaml"
$SUDO curl -fSLs "https://raw.githubusercontent.com/openfaas/faasd/${version}/prometheus.yml" --output "prometheus.yml"
$SUDO curl -fSLs "https://raw.githubusercontent.com/openfaas/faasd/${version}/resolv.conf" --output "resolv.conf"
$SUDO curl -fSLs "https://raw.githubusercontent.com/openfaas/faasd/${version}/hack/faasd-provider.service" --output "hack/faasd-provider.service"
$SUDO curl -fSLs "https://raw.githubusercontent.com/openfaas/faasd/${version}/hack/faasd.service" --output "hack/faasd.service"
$SUDO /usr/local/bin/faasd install
}
install_caddy() {
if [ ! -z "${FAASD_DOMAIN}" ]; then
CADDY_VER=v2.4.3
arkade get --progress=false caddy -v ${CADDY_VER}
# /usr/bin/caddy is specified in the upstream service file.
$SUDO install -m 755 $HOME/.arkade/bin/caddy /usr/bin/caddy
$SUDO curl -fSLs https://raw.githubusercontent.com/caddyserver/dist/master/init/caddy.service --output /etc/systemd/system/caddy.service
$SUDO mkdir -p /etc/caddy
$SUDO mkdir -p /var/lib/caddy
if $(id caddy >/dev/null 2>&1); then
echo "User caddy already exists."
else
$SUDO useradd --system --home /var/lib/caddy --shell /bin/false caddy
fi
$SUDO tee /etc/caddy/Caddyfile >/dev/null <<EOF
{
email "${LETSENCRYPT_EMAIL}"
}
${FAASD_DOMAIN} {
reverse_proxy 127.0.0.1:8080
}
EOF
$SUDO chown --recursive caddy:caddy /var/lib/caddy
$SUDO chown --recursive caddy:caddy /etc/caddy
$SUDO systemctl enable caddy
$SUDO systemctl start caddy
else
echo "Skipping caddy installation as FAASD_DOMAIN."
fi
}
install_faas_cli() {
arkade get --progress=false faas-cli
$SUDO install -m 755 $HOME/.arkade/bin/faas-cli /usr/local/bin/
}
verify_system
install_required_packages
$SUDO /sbin/sysctl -w net.ipv4.conf.all.forwarding=1
echo "net.ipv4.conf.all.forwarding=1" | $SUDO tee -a /etc/sysctl.conf
install_arkade
install_cni_plugins
install_containerd
install_faas_cli
install_faasd
install_caddy

10
main.go
View File

@ -7,6 +7,14 @@ import (
"github.com/openfaas/faasd/cmd"
)
// These values will be injected into these variables at the build time.
var (
// GitCommit Git Commit SHA
GitCommit string
// Version version of the CLI
Version string
)
func main() {
if _, ok := os.LookupEnv("CONTAINER_ID"); ok {
@ -23,7 +31,7 @@ func main() {
os.Exit(0)
}
if err := cmd.Execute(); err != nil {
if err := cmd.Execute(Version, GitCommit); err != nil {
os.Exit(1)
}
return

View File

@ -1,16 +1,15 @@
package cninetwork
import (
"bufio"
"context"
"fmt"
"io"
"io/ioutil"
"log"
"net"
"os"
"path"
"path/filepath"
"strings"
"github.com/containerd/containerd"
gocni "github.com/containerd/go-cni"
@ -20,31 +19,21 @@ import (
const (
// CNIBinDir describes the directory where the CNI binaries are stored
CNIBinDir = "/opt/cni/bin"
// CNIConfDir describes the directory where the CNI plugin's configuration is stored
CNIConfDir = "/etc/cni/net.d"
// NetNSPathFmt gives the path to the a process network namespace, given the pid
NetNSPathFmt = "/proc/%d/ns/net"
// CNIDataDir is the directory CNI stores allocated IP for containers
CNIDataDir = "/var/run/cni"
// CNIResultsDir is the directory CNI stores allocated IP for containers
CNIResultsDir = "/var/lib/cni/results"
// defaultCNIConfFilename is the vanity filename of default CNI configuration file
defaultCNIConfFilename = "10-openfaas.conflist"
// defaultNetworkName names the "docker-bridge"-like CNI plugin-chain installed when no other CNI configuration is present.
// This value appears in iptables comments created by CNI.
defaultNetworkName = "openfaas-cni-bridge"
// defaultBridgeName is the default bridge device name used in the defaultCNIConf
defaultBridgeName = "openfaas0"
// defaultSubnet is the default subnet used in the defaultCNIConf -- this value is set to not collide with common container networking subnets:
defaultSubnet = "10.62.0.0/16"
// defaultIfPrefix is the interface name to be created in the container
defaultIfPrefix = "eth"
)
// defaultCNIConf is a CNI configuration that enables network access to containers (docker-bridge style)
@ -61,7 +50,6 @@ var defaultCNIConf = fmt.Sprintf(`
"ipam": {
"type": "host-local",
"subnet": "%s",
"dataDir": "%s",
"routes": [
{ "dst": "0.0.0.0/0" }
]
@ -72,7 +60,7 @@ var defaultCNIConf = fmt.Sprintf(`
}
]
}
`, defaultNetworkName, defaultBridgeName, defaultSubnet, CNIDataDir)
`, defaultNetworkName, defaultBridgeName, defaultSubnet)
// InitNetwork writes configlist file and initializes CNI network
func InitNetwork() (gocni.CNI, error) {
@ -85,16 +73,13 @@ func InitNetwork() (gocni.CNI, error) {
}
netConfig := path.Join(CNIConfDir, defaultCNIConfFilename)
if err := os.WriteFile(netConfig, []byte(defaultCNIConf), 644); err != nil {
if err := ioutil.WriteFile(netConfig, []byte(defaultCNIConf), 644); err != nil {
return nil, fmt.Errorf("cannot write network config: %s", defaultCNIConfFilename)
}
}
// Initialize CNI library
cni, err := gocni.New(
gocni.WithPluginConfDir(CNIConfDir),
gocni.WithPluginDir([]string{CNIBinDir}),
gocni.WithInterfacePrefix(defaultIfPrefix),
)
cni, err := gocni.New(gocni.WithPluginConfDir(CNIConfDir),
gocni.WithPluginDir([]string{CNIBinDir}))
if err != nil {
return nil, fmt.Errorf("error initializing cni: %s", err)
@ -146,61 +131,43 @@ func DeleteCNINetwork(ctx context.Context, cni gocni.CNI, client *containerd.Cli
return errors.Wrapf(containerErr, "Unable to find container: %s, error: %s", name, containerErr)
}
// GetIPAddress returns the IP address from container based on container name and PID
func GetIPAddress(container string, PID uint32) (string, error) {
CNIDir := path.Join(CNIDataDir, defaultNetworkName)
files, err := os.ReadDir(CNIDir)
if err != nil {
return "", fmt.Errorf("failed to read CNI dir for container %s: %v", container, err)
}
for _, file := range files {
// each fileName is an IP address
fileName := file.Name()
resultsFile := filepath.Join(CNIDir, fileName)
found, err := isCNIResultForPID(resultsFile, container, PID)
if err != nil {
return "", err
}
if found {
return fileName, nil
// GetIPAddress returns the IP address of the created container
func GetIPAddress(result *gocni.CNIResult, task containerd.Task) (net.IP, error) {
// Get the IP of the created interface
var ip net.IP
for ifName, config := range result.Interfaces {
if config.Sandbox == netNamespace(task) {
for _, ipConfig := range config.IPConfigs {
if ifName != "lo" && ipConfig.IP.To4() != nil {
ip = ipConfig.IP
}
}
}
}
return "", fmt.Errorf("unable to get IP address for container: %s", container)
if ip == nil {
return nil, fmt.Errorf("unable to get IP address for: %s", task.ID())
}
return ip, nil
}
// isCNIResultForPID confirms if the CNI result file contains the
// process name, PID and interface name
//
// Example:
//
// /var/run/cni/openfaas-cni-bridge/10.62.0.2
//
// nats-621
// eth1
func isCNIResultForPID(fileName, container string, PID uint32) (bool, error) {
found := false
func GetIPfromPID(pid int) (*net.IP, error) {
// https://github.com/weaveworks/weave/blob/master/net/netdev.go
f, err := os.Open(fileName)
peerIDs, err := ConnectedToBridgeVethPeerIds(defaultBridgeName)
if err != nil {
return false, fmt.Errorf("failed to open CNI IP file for %s: %v", fileName, err)
}
defer f.Close()
reader := bufio.NewReader(f)
processLine, _ := reader.ReadString('\n')
if strings.Contains(processLine, fmt.Sprintf("%s-%d", container, PID)) {
ethNameLine, _ := reader.ReadString('\n')
if strings.Contains(ethNameLine, defaultIfPrefix) {
found = true
}
return nil, fmt.Errorf("unable to find peers on: %s %s", defaultBridgeName, err)
}
return found, nil
addrs, addrsErr := GetNetDevsByVethPeerIds(pid, peerIDs)
if addrsErr != nil {
return nil, fmt.Errorf("unable to find address for veth pair using: %v %s", peerIDs, addrsErr)
}
if len(addrs) > 0 && len(addrs[0].CIDRs) > 0 {
return &addrs[0].CIDRs[0].IP, nil
}
return nil, fmt.Errorf("no IP found for function")
}
// CNIGateway returns the gateway for default subnet

View File

@ -1,61 +0,0 @@
package cninetwork
import (
"os"
"path/filepath"
"testing"
)
func Test_isCNIResultForPID_Found(t *testing.T) {
body := `nats-621
eth1`
fileName := `10.62.0.2`
container := "nats"
PID := uint32(621)
fullPath := filepath.Join(os.TempDir(), fileName)
err := os.WriteFile(fullPath, []byte(body), 0700)
if err != nil {
t.Fatalf(err.Error())
}
defer func() {
os.Remove(fullPath)
}()
got, err := isCNIResultForPID(fullPath, container, PID)
if err != nil {
t.Fatalf(err.Error())
}
want := true
if got != want {
t.Fatalf("want %v, but got %v", want, got)
}
}
func Test_isCNIResultForPID_NoMatch(t *testing.T) {
body := `nats-621
eth1`
fileName := `10.62.0.3`
container := "gateway"
PID := uint32(621)
fullPath := filepath.Join(os.TempDir(), fileName)
err := os.WriteFile(fullPath, []byte(body), 0700)
if err != nil {
t.Fatalf(err.Error())
}
defer func() {
os.Remove(fullPath)
}()
got, err := isCNIResultForPID(fullPath, container, PID)
if err != nil {
t.Fatalf(err.Error())
}
want := false
if got != want {
t.Fatalf("want %v, but got %v", want, got)
}
}

118
pkg/cninetwork/weave.go Normal file
View File

@ -0,0 +1,118 @@
// Copyright Weaveworks
// github.com/weaveworks/weave/net
package cninetwork
import (
"fmt"
"net"
"os"
"github.com/vishvananda/netlink"
"github.com/vishvananda/netns"
)
type Dev struct {
Name string `json:"Name,omitempty"`
MAC net.HardwareAddr `json:"MAC,omitempty"`
CIDRs []*net.IPNet `json:"CIDRs,omitempty"`
}
// ConnectedToBridgeVethPeerIds returns peer indexes of veth links connected to
// the given bridge. The peer index is used to query from a container netns
// whether the container is connected to the bridge.
func ConnectedToBridgeVethPeerIds(bridgeName string) ([]int, error) {
var ids []int
br, err := netlink.LinkByName(bridgeName)
if err != nil {
return nil, err
}
links, err := netlink.LinkList()
if err != nil {
return nil, err
}
for _, link := range links {
if _, isveth := link.(*netlink.Veth); isveth && link.Attrs().MasterIndex == br.Attrs().Index {
peerID := link.Attrs().ParentIndex
if peerID == 0 {
// perhaps running on an older kernel where ParentIndex doesn't work.
// as fall-back, assume the peers are consecutive
peerID = link.Attrs().Index - 1
}
ids = append(ids, peerID)
}
}
return ids, nil
}
// Lookup the weave interface of a container
func GetWeaveNetDevs(processID int) ([]Dev, error) {
peerIDs, err := ConnectedToBridgeVethPeerIds("weave")
if err != nil {
return nil, err
}
return GetNetDevsByVethPeerIds(processID, peerIDs)
}
func GetNetDevsByVethPeerIds(processID int, peerIDs []int) ([]Dev, error) {
// Bail out if this container is running in the root namespace
netnsRoot, err := netns.GetFromPid(1)
if err != nil {
return nil, fmt.Errorf("unable to open root namespace: %s", err)
}
defer netnsRoot.Close()
netnsContainer, err := netns.GetFromPid(processID)
if err != nil {
// Unable to find a namespace for this process - just return nothing
if os.IsNotExist(err) {
return nil, nil
}
return nil, fmt.Errorf("unable to open process %d namespace: %s", processID, err)
}
defer netnsContainer.Close()
if netnsRoot.Equal(netnsContainer) {
return nil, nil
}
// convert list of peerIDs into a map for faster lookup
indexes := make(map[int]struct{})
for _, id := range peerIDs {
indexes[id] = struct{}{}
}
var netdevs []Dev
err = WithNetNS(netnsContainer, func() error {
links, err := netlink.LinkList()
if err != nil {
return err
}
for _, link := range links {
if _, found := indexes[link.Attrs().Index]; found {
netdev, err := linkToNetDev(link)
if err != nil {
return err
}
netdevs = append(netdevs, netdev)
}
}
return nil
})
return netdevs, err
}
// Get the weave bridge interface.
// NB: Should be called from the root network namespace.
func GetBridgeNetDev(bridgeName string) (Dev, error) {
link, err := netlink.LinkByName(bridgeName)
if err != nil {
return Dev{}, err
}
return linkToNetDev(link)
}

View File

@ -0,0 +1,10 @@
// +build darwin
package cninetwork
import "github.com/vishvananda/netlink"
func linkToNetDev(link netlink.Link) (Dev, error) {
return Dev{}, nil
}

View File

@ -0,0 +1,19 @@
// +build linux
package cninetwork
import "github.com/vishvananda/netlink"
func linkToNetDev(link netlink.Link) (Dev, error) {
addrs, err := netlink.AddrList(link, netlink.FAMILY_V4)
if err != nil {
return Dev{}, err
}
netDev := Dev{Name: link.Attrs().Name, MAC: link.Attrs().HardwareAddr}
for _, addr := range addrs {
netDev.CIDRs = append(netDev.CIDRs, addr.IPNet)
}
return netDev, nil
}

View File

@ -1,39 +0,0 @@
package pkg
import (
"fmt"
"io"
"net/http"
"strings"
)
// ConnectivityCheck checks if the controller can reach the
// public Internet via HTTPS.
// A license is required to use OpenFaaS CE for Commercial Use.
func ConnectivityCheck() error {
req, err := http.NewRequest(http.MethodGet, "https://checkip.amazonaws.com", nil)
if err != nil {
return err
}
req.Header.Set("User-Agent", fmt.Sprintf("openfaas-ce/%s faas-netes", Version))
res, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
if req.Body != nil {
defer req.Body.Close()
}
if res.StatusCode != http.StatusOK {
var body []byte
if res.Body != nil {
body, _ = io.ReadAll(res.Body)
}
return fmt.Errorf("unexpected status code checking connectivity: %d, body: %s", res.StatusCode, strings.TrimSpace(string(body)))
}
return nil
}

View File

@ -1,16 +0,0 @@
package pkg
const (
// DefaultFunctionNamespace is the default containerd namespace functions are created
DefaultFunctionNamespace = "openfaas-fn"
// NamespaceLabel indicates that a namespace is managed by faasd
NamespaceLabel = "openfaas"
// FaasdNamespace is the containerd namespace services are created
FaasdNamespace = "openfaas"
faasServicesPullAlways = false
defaultSnapshotter = "overlayfs"
)

6
pkg/contants.go Normal file
View File

@ -0,0 +1,6 @@
package pkg
const (
// FunctionNamespace is the default containerd namespace functions are created
FunctionNamespace = "openfaas-fn"
)

View File

@ -1,103 +0,0 @@
package pkg
import (
"log"
"os"
"strings"
"sync"
"time"
)
// LocalResolver provides hostname to IP look-up for faasd core services
type LocalResolver struct {
Path string
Map map[string]string
Mutex *sync.RWMutex
}
// NewLocalResolver creates a new resolver for reading from a hosts file
func NewLocalResolver(path string) Resolver {
return &LocalResolver{
Path: path,
Mutex: &sync.RWMutex{},
Map: make(map[string]string),
}
}
// Start polling the disk for the hosts file in Path
func (l *LocalResolver) Start() {
var lastStat os.FileInfo
for {
rebuild := false
if info, err := os.Stat(l.Path); err == nil {
if lastStat == nil {
rebuild = true
} else {
if !lastStat.ModTime().Equal(info.ModTime()) {
rebuild = true
}
}
lastStat = info
}
if rebuild {
log.Printf("Resolver rebuilding map")
l.rebuild()
}
time.Sleep(time.Second * 3)
}
}
func (l *LocalResolver) rebuild() {
l.Mutex.Lock()
defer l.Mutex.Unlock()
fileData, fileErr := os.ReadFile(l.Path)
if fileErr != nil {
log.Printf("resolver rebuild error: %s", fileErr.Error())
return
}
lines := strings.Split(string(fileData), "\n")
for _, line := range lines {
index := strings.Index(line, "\t")
if len(line) > 0 && index > -1 {
ip := line[:index]
host := line[index+1:]
log.Printf("Resolver: %q=%q", host, ip)
l.Map[host] = ip
}
}
}
// Get resolves a hostname to an IP, or timesout after the duration has passed
func (l *LocalResolver) Get(upstream string, got chan<- string, timeout time.Duration) {
start := time.Now()
for {
if val := l.get(upstream); len(val) > 0 {
got <- val
break
}
if time.Now().After(start.Add(timeout)) {
log.Printf("Timed out after %s getting host %q", timeout.String(), upstream)
break
}
time.Sleep(time.Millisecond * 250)
}
}
func (l *LocalResolver) get(upstream string) string {
l.Mutex.RLock()
defer l.Mutex.RUnlock()
if val, ok := l.Map[upstream]; ok {
return val
}
return ""
}

View File

@ -58,10 +58,10 @@ func (r *requester) Query(ctx context.Context, req logs.Request) (<-chan logs.Me
// buildCmd reeturns the equivalent of
//
// journalctl -t <namespace>:<name> \
// --output=json \
// --since=<timestamp> \
// <--follow> \
// journalctl -t <namespace>:<name> \
// --output=json \
// --since=<timestamp> \
// <--follow> \
func buildCmd(ctx context.Context, req logs.Request) *exec.Cmd {
// // set the cursor position based on req, default to 5m
since := time.Now().Add(-5 * time.Minute)
@ -71,7 +71,7 @@ func buildCmd(ctx context.Context, req logs.Request) *exec.Cmd {
namespace := req.Namespace
if namespace == "" {
namespace = faasd.DefaultFunctionNamespace
namespace = faasd.FunctionNamespace
}
// find the description of the fields here
@ -105,12 +105,12 @@ func streamLogs(ctx context.Context, cmd *exec.Cmd, out io.ReadCloser, msgs chan
// will ensure `out` is closed and all related resources cleaned up
go func() {
if err := cmd.Wait(); err != nil {
log.Printf("journalctl exited with error: %s", err)
}
err := cmd.Wait()
log.Println("wait result", err)
}()
defer func() {
log.Println("closing journal stream")
close(msgs)
}()
@ -176,6 +176,7 @@ func parseEntry(entry map[string]string) (logs.Message, error) {
}
func logErrOut(out io.ReadCloser) {
defer log.Println("stderr closed")
defer out.Close()
io.Copy(log.Writer(), out)

View File

@ -20,11 +20,9 @@ func ReadFromEnv(hasEnv types.HasEnv) (*types.FaaSConfig, *ProviderConfig, error
serviceTimeout := types.ParseIntOrDurationValue(hasEnv.Getenv("service_timeout"), time.Second*60)
config.EnableHealth = true
config.ReadTimeout = serviceTimeout
config.WriteTimeout = serviceTimeout
config.EnableBasicAuth = true
config.MaxIdleConns = types.ParseIntValue(hasEnv.Getenv("max_idle_conns"), 1024)
config.MaxIdleConnsPerHost = types.ParseIntValue(hasEnv.Getenv("max_idle_conns_per_host"), 1024)
port := types.ParseIntValue(hasEnv.Getenv("port"), 8081)
config.TCPPort = &port

View File

@ -4,16 +4,16 @@ import (
"context"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"github.com/containerd/containerd"
"github.com/containerd/containerd/namespaces"
gocni "github.com/containerd/go-cni"
"github.com/openfaas/faas/gateway/requests"
"github.com/openfaas/faas-provider/types"
"github.com/openfaas/faasd/pkg"
faasd "github.com/openfaas/faasd/pkg"
cninetwork "github.com/openfaas/faasd/pkg/cninetwork"
"github.com/openfaas/faasd/pkg/service"
)
@ -29,45 +29,29 @@ func MakeDeleteHandler(client *containerd.Client, cni gocni.CNI) func(w http.Res
defer r.Body.Close()
body, _ := io.ReadAll(r.Body)
body, _ := ioutil.ReadAll(r.Body)
log.Printf("[Delete] request: %s\n", string(body))
req := types.DeleteFunctionRequest{}
if err := json.Unmarshal(body, &req); err != nil {
log.Printf("[Delete] error parsing input: %s", err)
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
// namespace moved from the querystring into the body
namespace := req.Namespace
if namespace == "" {
namespace = pkg.DefaultFunctionNamespace
}
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client.NamespaceService(), namespace)
req := requests.DeleteFunctionRequest{}
err := json.Unmarshal(body, &req)
if err != nil {
log.Printf("[Delete] error parsing input: %s\n", err)
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
name := req.FunctionName
function, err := GetFunction(client, name, namespace)
function, err := GetFunction(client, name)
if err != nil {
msg := fmt.Sprintf("function %s.%s not found", name, namespace)
msg := fmt.Sprintf("service %s not found", name)
log.Printf("[Delete] %s\n", msg)
http.Error(w, msg, http.StatusNotFound)
return
}
ctx := namespaces.WithNamespace(context.Background(), namespace)
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace)
// TODO: this needs to still happen if the task is paused
if function.replicas != 0 {
@ -77,12 +61,13 @@ func MakeDeleteHandler(client *containerd.Client, cni gocni.CNI) func(w http.Res
}
}
if err := service.Remove(ctx, client, name); err != nil {
log.Printf("[Delete] error removing %s, %s\n", name, err)
http.Error(w, err.Error(), http.StatusInternalServerError)
containerErr := service.Remove(ctx, client, name)
if containerErr != nil {
log.Printf("[Delete] error removing %s, %s\n", name, containerErr)
http.Error(w, containerErr.Error(), http.StatusInternalServerError)
return
}
log.Printf("[Delete] Removed: %s.%s\n", name, namespace)
log.Printf("[Delete] deleted %s\n", name)
}
}

View File

@ -4,32 +4,28 @@ import (
"context"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"path"
"time"
"github.com/containerd/containerd"
"github.com/containerd/containerd/cio"
"github.com/containerd/containerd/containers"
"github.com/containerd/containerd/namespaces"
"github.com/containerd/containerd/oci"
gocni "github.com/containerd/go-cni"
"github.com/distribution/reference"
"github.com/docker/distribution/reference"
"github.com/opencontainers/runtime-spec/specs-go"
"github.com/openfaas/faas-provider/types"
faasd "github.com/openfaas/faasd/pkg"
cninetwork "github.com/openfaas/faasd/pkg/cninetwork"
"github.com/openfaas/faasd/pkg/service"
"github.com/pkg/errors"
"k8s.io/apimachinery/pkg/api/resource"
)
const annotationLabelPrefix = "com.openfaas.annotations."
// MakeDeployHandler returns a handler to deploy a function
func MakeDeployHandler(client *containerd.Client, cni gocni.CNI, secretMountPath string, alwaysPull bool) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
if r.Body == nil {
@ -39,66 +35,40 @@ func MakeDeployHandler(client *containerd.Client, cni gocni.CNI, secretMountPath
defer r.Body.Close()
body, _ := io.ReadAll(r.Body)
body, _ := ioutil.ReadAll(r.Body)
log.Printf("[Deploy] request: %s\n", string(body))
req := types.FunctionDeployment{}
err := json.Unmarshal(body, &req)
if err != nil {
log.Printf("[Deploy] - error parsing input: %s", err)
log.Printf("[Deploy] - error parsing input: %s\n", err)
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
namespace := getRequestNamespace(req.Namespace)
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client.NamespaceService(), namespace)
err = validateSecrets(secretMountPath, req.Secrets)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
namespaceSecretMountPath := getNamespaceSecretMountPath(secretMountPath, namespace)
err = validateSecrets(namespaceSecretMountPath, req.Secrets)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
name := req.Service
ctx := namespaces.WithNamespace(context.Background(), namespace)
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace)
if err := preDeploy(client, 1); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
log.Printf("[Deploy] error deploying %s, error: %s\n", name, err)
return
}
if err := deploy(ctx, req, client, cni, namespaceSecretMountPath, alwaysPull); err != nil {
log.Printf("[Deploy] error deploying %s, error: %s\n", name, err)
http.Error(w, err.Error(), http.StatusBadRequest)
deployErr := deploy(ctx, req, client, cni, secretMountPath, alwaysPull)
if deployErr != nil {
log.Printf("[Deploy] error deploying %s, error: %s\n", name, deployErr)
http.Error(w, deployErr.Error(), http.StatusBadRequest)
return
}
}
}
// prepull is an optimization which means an image can be pulled before a deployment
// request, since a deployment request first deletes the active function before
// trying to deploy a new one.
func prepull(ctx context.Context, req types.FunctionDeployment, client *containerd.Client, alwaysPull bool) (containerd.Image, error) {
start := time.Now()
func deploy(ctx context.Context, req types.FunctionDeployment, client *containerd.Client, cni gocni.CNI, secretMountPath string, alwaysPull bool) error {
r, err := reference.ParseNormalizedNamed(req.Image)
if err != nil {
return nil, err
return err
}
imgRef := reference.TagNameOnly(r).String()
snapshotter := ""
@ -108,29 +78,14 @@ func prepull(ctx context.Context, req types.FunctionDeployment, client *containe
image, err := service.PrepareImage(ctx, client, imgRef, snapshotter, alwaysPull)
if err != nil {
return nil, errors.Wrapf(err, "unable to pull image %s", imgRef)
return errors.Wrapf(err, "unable to pull image %s", imgRef)
}
size, _ := image.Size(ctx)
log.Printf("Image for: %s size: %d, took: %fs\n", image.Name(), size, time.Since(start).Seconds())
return image, nil
}
func deploy(ctx context.Context, req types.FunctionDeployment, client *containerd.Client, cni gocni.CNI, secretMountPath string, alwaysPull bool) error {
snapshotter := ""
if val, ok := os.LookupEnv("snapshotter"); ok {
snapshotter = val
}
image, err := prepull(ctx, req, client, alwaysPull)
if err != nil {
return err
}
log.Printf("Deploy %s size: %d\n", image.Name(), size)
envs := prepareEnv(req.EnvProcess, req.EnvVars)
mounts := getOSMounts()
mounts := getMounts()
for _, secret := range req.Secrets {
mounts = append(mounts, specs.Mount{
@ -143,23 +98,6 @@ func deploy(ctx context.Context, req types.FunctionDeployment, client *container
name := req.Service
labels, err := buildLabels(&req)
if err != nil {
return fmt.Errorf("unable to apply labels to container: %s, error: %w", name, err)
}
var memory *specs.LinuxMemory
if req.Limits != nil && len(req.Limits.Memory) > 0 {
memory = &specs.LinuxMemory{}
qty, err := resource.ParseQuantity(req.Limits.Memory)
if err != nil {
log.Printf("error parsing (%q) as quantity: %s", req.Limits.Memory, err.Error())
}
v := qty.Value()
memory.Limit = &v
}
container, err := client.NewContainer(
ctx,
name,
@ -167,93 +105,49 @@ func deploy(ctx context.Context, req types.FunctionDeployment, client *container
containerd.WithSnapshotter(snapshotter),
containerd.WithNewSnapshot(name+"-snapshot", image),
containerd.WithNewSpec(oci.WithImageConfig(image),
oci.WithHostname(name),
oci.WithCapabilities([]string{"CAP_NET_RAW"}),
oci.WithMounts(mounts),
oci.WithEnv(envs),
withMemory(memory)),
containerd.WithContainerLabels(labels),
oci.WithEnv(envs)),
containerd.WithContainerLabels(*req.Labels),
)
if err != nil {
return fmt.Errorf("unable to create container: %s, error: %w", name, err)
return fmt.Errorf("unable to create container: %s, error: %s", name, err)
}
return createTask(ctx, container, cni)
return createTask(ctx, client, container, cni)
}
// countFunctions returns the number of functions deployed along with a map with a count
// in each namespace
func countFunctions(client *containerd.Client) (int64, int64, error) {
count := int64(0)
namespaceCount := int64(0)
namespaces := ListNamespaces(client)
for _, namespace := range namespaces {
fns, err := ListFunctions(client, namespace)
if err != nil {
return 0, 0, err
}
namespaceCount++
count += int64(len(fns))
}
return count, namespaceCount, nil
}
func buildLabels(request *types.FunctionDeployment) (map[string]string, error) {
labels := map[string]string{}
if request.Labels != nil {
for k, v := range *request.Labels {
labels[k] = v
}
}
if request.Annotations != nil {
for k, v := range *request.Annotations {
key := fmt.Sprintf("%s%s", annotationLabelPrefix, k)
if _, ok := labels[key]; !ok {
labels[key] = v
} else {
return nil, errors.New(fmt.Sprintf("Key %s cannot be used as a label due to a conflict with annotation prefix %s", k, annotationLabelPrefix))
}
}
}
return labels, nil
}
func createTask(ctx context.Context, container containerd.Container, cni gocni.CNI) error {
func createTask(ctx context.Context, client *containerd.Client, container containerd.Container, cni gocni.CNI) error {
name := container.ID()
// task, taskErr := container.NewTask(ctx, cio.NewCreator(cio.WithStdio))
task, taskErr := container.NewTask(ctx, cio.BinaryIO("/usr/local/bin/faasd", nil))
if taskErr != nil {
return fmt.Errorf("unable to start task: %s, error: %w", name, taskErr)
return fmt.Errorf("unable to start task: %s, error: %s", name, taskErr)
}
log.Printf("Container ID: %s\tTask ID %s:\tTask PID: %d\t\n", name, task.ID(), task.Pid())
labels := map[string]string{}
_, err := cninetwork.CreateCNINetwork(ctx, cni, task, labels)
network, err := cninetwork.CreateCNINetwork(ctx, cni, task, labels)
if err != nil {
return err
}
ip, err := cninetwork.GetIPAddress(name, task.Pid())
ip, err := cninetwork.GetIPAddress(network, task)
if err != nil {
return err
}
log.Printf("%s has IP: %s.\n", name, ip.String())
log.Printf("%s has IP: %s.\n", name, ip)
if _, err := task.Wait(ctx); err != nil {
return errors.Wrapf(err, "Unable to wait for task to start: %s", name)
_, waitErr := task.Wait(ctx)
if waitErr != nil {
return errors.Wrapf(waitErr, "Unable to wait for task to start: %s", name)
}
if startErr := task.Start(ctx); startErr != nil {
@ -284,28 +178,20 @@ func prepareEnv(envProcess string, reqEnvVars map[string]string) []string {
return envs
}
// getOSMounts provides a mount for os-specific files such
// as the hosts file and resolv.conf
func getOSMounts() []specs.Mount {
// Prior to hosts_dir env-var, this value was set to
// os.Getwd()
hostsDir := "/var/lib/faasd"
if v, ok := os.LookupEnv("hosts_dir"); ok && len(v) > 0 {
hostsDir = v
}
func getMounts() []specs.Mount {
wd, _ := os.Getwd()
mounts := []specs.Mount{}
mounts = append(mounts, specs.Mount{
Destination: "/etc/resolv.conf",
Type: "bind",
Source: path.Join(hostsDir, "resolv.conf"),
Source: path.Join(wd, "resolv.conf"),
Options: []string{"rbind", "ro"},
})
mounts = append(mounts, specs.Mount{
Destination: "/etc/hosts",
Type: "bind",
Source: path.Join(hostsDir, "hosts"),
Source: path.Join(wd, "hosts"),
Options: []string{"rbind", "ro"},
})
return mounts
@ -319,35 +205,3 @@ func validateSecrets(secretMountPath string, secrets []string) error {
}
return nil
}
func withMemory(mem *specs.LinuxMemory) oci.SpecOpts {
return func(ctx context.Context, _ oci.Client, c *containers.Container, s *oci.Spec) error {
if mem != nil {
if s.Linux == nil {
s.Linux = &specs.Linux{}
}
if s.Linux.Resources == nil {
s.Linux.Resources = &specs.LinuxResources{}
}
if s.Linux.Resources.Memory == nil {
s.Linux.Resources.Memory = &specs.LinuxMemory{}
}
s.Linux.Resources.Memory.Limit = mem.Limit
}
return nil
}
}
func preDeploy(client *containerd.Client, additional int64) error {
count, countNs, err := countFunctions(client)
log.Printf("Function count: %d, Namespace count: %d\n", count, countNs)
if err != nil {
return err
} else if count+additional > faasdMaxFunctions {
return fmt.Errorf("the OpenFaaS CE EULA allows %d/%d function(s), upgrade to faasd Pro to continue", faasdMaxFunctions, count+additional)
} else if countNs > faasdMaxNs {
return fmt.Errorf("the OpenFaaS CE EULA allows %d/%d namespace(s), upgrade to faasd Pro to continue", faasdMaxNs, countNs)
}
return nil
}

View File

@ -1,77 +0,0 @@
package handlers
import (
"fmt"
"reflect"
"testing"
"github.com/openfaas/faas-provider/types"
)
func Test_BuildLabels_WithAnnotations(t *testing.T) {
// Test each combination of nil/non-nil annotation + label
tables := []struct {
name string
label map[string]string
annotation map[string]string
result map[string]string
}{
{"Empty label and annotations returns empty table map", nil, nil, map[string]string{}},
{
"Label with empty annotation returns valid map",
map[string]string{"L1": "V1"},
nil,
map[string]string{"L1": "V1"}},
{
"Annotation with empty label returns valid map",
nil,
map[string]string{"A1": "V2"},
map[string]string{fmt.Sprintf("%sA1", annotationLabelPrefix): "V2"}},
{
"Label and annotation provided returns valid combined map",
map[string]string{"L1": "V1"},
map[string]string{"A1": "V2"},
map[string]string{
"L1": "V1",
fmt.Sprintf("%sA1", annotationLabelPrefix): "V2",
},
},
}
for _, tc := range tables {
t.Run(tc.name, func(t *testing.T) {
request := &types.FunctionDeployment{
Labels: &tc.label,
Annotations: &tc.annotation,
}
val, err := buildLabels(request)
if err != nil {
t.Fatalf("want: no error got: %v", err)
}
if !reflect.DeepEqual(val, tc.result) {
t.Errorf("Want: %s, got: %s", val, tc.result)
}
})
}
}
func Test_BuildLabels_WithAnnotationCollision(t *testing.T) {
request := &types.FunctionDeployment{
Labels: &map[string]string{
"function_name": "echo",
fmt.Sprintf("%scurrent-time", annotationLabelPrefix): "Wed 25 Jul 06:41:43 BST 2018",
},
Annotations: &map[string]string{"current-time": "Wed 25 Jul 06:41:43 BST 2018"},
}
val, err := buildLabels(request)
if err == nil {
t.Errorf("Expected an error, got %d values", len(val))
}
}

View File

@ -1,150 +0,0 @@
package handlers
import (
"context"
"fmt"
"log"
"strings"
"github.com/opencontainers/runtime-spec/specs-go"
"github.com/containerd/containerd"
"github.com/containerd/containerd/namespaces"
"github.com/openfaas/faasd/pkg/cninetwork"
)
// GetFunction returns a function that matches name
func GetFunction(client *containerd.Client, name string, namespace string) (Function, error) {
ctx := namespaces.WithNamespace(context.Background(), namespace)
fn := Function{}
c, err := client.LoadContainer(ctx, name)
if err != nil {
return Function{}, fmt.Errorf("unable to find function: %s, error %w", name, err)
}
image, err := c.Image(ctx)
if err != nil {
return fn, err
}
containerName := c.ID()
allLabels, labelErr := c.Labels(ctx)
if labelErr != nil {
log.Printf("cannot list container %s labels: %s", containerName, labelErr)
}
labels, annotations := buildLabelsAndAnnotations(allLabels)
spec, err := c.Spec(ctx)
if err != nil {
return Function{}, fmt.Errorf("unable to load function %s error: %w", name, err)
}
info, err := c.Info(ctx)
if err != nil {
return Function{}, fmt.Errorf("can't load info for: %s, error %w", name, err)
}
envVars, envProcess := readEnvFromProcessEnv(spec.Process.Env)
secrets := readSecretsFromMounts(spec.Mounts)
fn.name = containerName
fn.namespace = namespace
fn.image = image.Name()
fn.labels = labels
fn.annotations = annotations
fn.secrets = secrets
fn.envVars = envVars
fn.envProcess = envProcess
fn.createdAt = info.CreatedAt
fn.memoryLimit = readMemoryLimitFromSpec(spec)
replicas := 0
task, err := c.Task(ctx, nil)
if err == nil {
// Task for container exists
svc, err := task.Status(ctx)
if err != nil {
return Function{}, fmt.Errorf("unable to get task status for container: %s %w", name, err)
}
if svc.Status == "running" {
replicas = 1
fn.pid = task.Pid()
// Get container IP address
ip, err := cninetwork.GetIPAddress(name, task.Pid())
if err != nil {
return Function{}, err
}
fn.IP = ip
}
} else {
replicas = 0
}
fn.replicas = replicas
return fn, nil
}
func readEnvFromProcessEnv(env []string) (map[string]string, string) {
foundEnv := make(map[string]string)
fprocess := ""
for _, e := range env {
kv := strings.Split(e, "=")
if len(kv) == 1 {
continue
}
if kv[0] == "PATH" {
continue
}
if kv[0] == "fprocess" {
fprocess = kv[1]
continue
}
foundEnv[kv[0]] = kv[1]
}
return foundEnv, fprocess
}
func readSecretsFromMounts(mounts []specs.Mount) []string {
secrets := []string{}
for _, mnt := range mounts {
x := strings.Split(mnt.Destination, "/var/openfaas/secrets/")
if len(x) > 1 {
secrets = append(secrets, x[1])
}
}
return secrets
}
// buildLabelsAndAnnotations returns a separated list with labels first,
// followed by annotations by checking each key of ctrLabels for a prefix.
func buildLabelsAndAnnotations(ctrLabels map[string]string) (map[string]string, map[string]string) {
labels := make(map[string]string)
annotations := make(map[string]string)
for k, v := range ctrLabels {
if strings.HasPrefix(k, annotationLabelPrefix) {
annotations[strings.TrimPrefix(k, annotationLabelPrefix)] = v
} else {
labels[k] = v
}
}
return labels, annotations
}
func readMemoryLimitFromSpec(spec *specs.Spec) int64 {
if spec.Linux == nil || spec.Linux.Resources == nil || spec.Linux.Resources.Memory == nil || spec.Linux.Resources.Memory.Limit == nil {
return 0
}
return *spec.Linux.Resources.Memory.Limit
}

View File

@ -1,65 +0,0 @@
package handlers
import (
"context"
"errors"
"log"
"strings"
"time"
"github.com/containerd/containerd"
"github.com/containerd/containerd/namespaces"
)
type Function struct {
name string
namespace string
image string
pid uint32
replicas int
IP string
labels map[string]string
annotations map[string]string
secrets []string
envVars map[string]string
envProcess string
memoryLimit int64
createdAt time.Time
}
// ListFunctions returns a map of all functions with running tasks on namespace
func ListFunctions(client *containerd.Client, namespace string) (map[string]*Function, error) {
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client.NamespaceService(), namespace)
if err != nil {
return nil, err
}
if !valid {
return nil, errors.New("namespace not valid")
}
ctx := namespaces.WithNamespace(context.Background(), namespace)
functions := make(map[string]*Function)
containers, err := client.Containers(ctx)
if err != nil {
return functions, err
}
for _, c := range containers {
name := c.ID()
f, err := GetFunction(client, name, namespace)
if err != nil {
if !strings.Contains(err.Error(), "unable to get IP address for container") {
log.Printf("List functions, skipping: %s, error: %s", name, err)
}
} else {
functions[name] = &f
}
}
return functions, nil
}

View File

@ -1,133 +0,0 @@
package handlers
import (
"fmt"
"reflect"
"testing"
"github.com/opencontainers/runtime-spec/specs-go"
)
func Test_BuildLabelsAndAnnotationsFromServiceSpec_Annotations(t *testing.T) {
container := map[string]string{
"qwer": "ty",
"dvor": "ak",
fmt.Sprintf("%scurrent-time", annotationLabelPrefix): "5 Nov 20:10:20 PST 1955",
fmt.Sprintf("%sfuture-time", annotationLabelPrefix): "21 Oct 20:10:20 PST 2015",
}
labels, annotation := buildLabelsAndAnnotations(container)
if len(labels) != 2 {
t.Errorf("want: %d labels got: %d", 2, len(labels))
}
if len(annotation) != 2 {
t.Errorf("want: %d annotation got: %d", 1, len(annotation))
}
if _, ok := annotation["current-time"]; !ok {
t.Errorf("want: '%s' entry in annotation map got: key not found", "current-time")
}
}
func Test_SplitMountToSecrets(t *testing.T) {
type testCase struct {
Name string
Input []specs.Mount
Want []string
}
tests := []testCase{
{Name: "No matching openfaas secrets", Input: []specs.Mount{{Destination: "/foo/"}}, Want: []string{}},
{Name: "Nil mounts", Input: nil, Want: []string{}},
{Name: "No Mounts", Input: []specs.Mount{{Destination: "/foo/"}}, Want: []string{}},
{Name: "One Mounts IS secret", Input: []specs.Mount{{Destination: "/var/openfaas/secrets/secret1"}}, Want: []string{"secret1"}},
{Name: "Multiple Mounts 1 secret", Input: []specs.Mount{{Destination: "/var/openfaas/secrets/secret1"}, {Destination: "/some/other/path"}}, Want: []string{"secret1"}},
{Name: "Multiple Mounts all secrets", Input: []specs.Mount{{Destination: "/var/openfaas/secrets/secret1"}, {Destination: "/var/openfaas/secrets/secret2"}}, Want: []string{"secret1", "secret2"}},
}
for _, tc := range tests {
t.Run(tc.Name, func(t *testing.T) {
got := readSecretsFromMounts(tc.Input)
if !reflect.DeepEqual(got, tc.Want) {
t.Fatalf("Want %s, got %s", tc.Want, got)
}
})
}
}
func Test_ProcessEnvToEnvVars(t *testing.T) {
type testCase struct {
Name string
Input []string
Want map[string]string
fprocess string
}
tests := []testCase{
{Name: "No matching EnvVars", Input: []string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "fprocess=python index.py"}, Want: make(map[string]string), fprocess: "python index.py"},
{Name: "No EnvVars", Input: []string{}, Want: make(map[string]string), fprocess: ""},
{Name: "One EnvVar", Input: []string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "fprocess=python index.py", "env=this"}, Want: map[string]string{"env": "this"}, fprocess: "python index.py"},
{Name: "Multiple EnvVars", Input: []string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "this=that", "env=var", "fprocess=python index.py"}, Want: map[string]string{"this": "that", "env": "var"}, fprocess: "python index.py"},
{Name: "Nil EnvVars", Input: nil, Want: make(map[string]string)},
}
for _, tc := range tests {
t.Run(tc.Name, func(t *testing.T) {
got, fprocess := readEnvFromProcessEnv(tc.Input)
if !reflect.DeepEqual(got, tc.Want) {
t.Fatalf("Want: %s, got: %s", tc.Want, got)
}
if fprocess != tc.fprocess {
t.Fatalf("Want fprocess: %s, got: %s", tc.fprocess, got)
}
})
}
}
func Test_findNamespace(t *testing.T) {
type testCase struct {
Name string
foundNamespaces []string
namespace string
Want bool
}
tests := []testCase{
{Name: "Namespace Found", namespace: "fn", foundNamespaces: []string{"fn", "openfaas-fn"}, Want: true},
{Name: "namespace Not Found", namespace: "fn", foundNamespaces: []string{"openfaas-fn"}, Want: false},
}
for _, tc := range tests {
t.Run(tc.Name, func(t *testing.T) {
got := findNamespace(tc.namespace, tc.foundNamespaces)
if got != tc.Want {
t.Fatalf("Want %t, got %t", tc.Want, got)
}
})
}
}
func Test_readMemoryLimitFromSpec(t *testing.T) {
type testCase struct {
Name string
Spec *specs.Spec
Want int64
}
testLimit := int64(64)
tests := []testCase{
{Name: "specs.Linux not found", Spec: &specs.Spec{Linux: nil}, Want: int64(0)},
{Name: "specs.LinuxResource not found", Spec: &specs.Spec{Linux: &specs.Linux{Resources: nil}}, Want: int64(0)},
{Name: "specs.LinuxMemory not found", Spec: &specs.Spec{Linux: &specs.Linux{Resources: &specs.LinuxResources{Memory: nil}}}, Want: int64(0)},
{Name: "specs.LinuxMemory.Limit not found", Spec: &specs.Spec{Linux: &specs.Linux{Resources: &specs.LinuxResources{Memory: &specs.LinuxMemory{Limit: nil}}}}, Want: int64(0)},
{Name: "Memory limit set as Want", Spec: &specs.Spec{Linux: &specs.Linux{Resources: &specs.LinuxResources{Memory: &specs.LinuxMemory{Limit: &testLimit}}}}, Want: int64(64)},
}
for _, tc := range tests {
t.Run(tc.Name, func(t *testing.T) {
got := readMemoryLimitFromSpec(tc.Spec)
if got != tc.Want {
t.Fatalf("Want %d, got %d", tc.Want, got)
}
})
}
}

View File

@ -0,0 +1,91 @@
package handlers
import (
"context"
"fmt"
"log"
"github.com/containerd/containerd"
"github.com/containerd/containerd/namespaces"
"github.com/openfaas/faasd/pkg/cninetwork"
faasd "github.com/openfaas/faasd/pkg"
)
type Function struct {
name string
namespace string
image string
pid uint32
replicas int
IP string
labels map[string]string
}
// ListFunctions returns a map of all functions with running tasks on namespace
func ListFunctions(client *containerd.Client) (map[string]Function, error) {
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace)
functions := make(map[string]Function)
containers, _ := client.Containers(ctx)
for _, k := range containers {
name := k.ID()
f, err := GetFunction(client, name)
if err != nil {
continue
}
functions[name] = f
}
return functions, nil
}
// GetFunction returns a function that matches name
func GetFunction(client *containerd.Client, name string) (Function, error) {
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace)
c, err := client.LoadContainer(ctx, name)
if err == nil {
image, _ := c.Image(ctx)
containerName := c.ID()
labels, labelErr := c.Labels(ctx)
if labelErr != nil {
log.Printf("cannot list container %s labels: %s", containerName, labelErr.Error())
}
f := Function{
name: containerName,
namespace: faasd.FunctionNamespace,
image: image.Name(),
labels: labels,
}
replicas := 0
task, err := c.Task(ctx, nil)
if err == nil {
// Task for container exists
svc, err := task.Status(ctx)
if err != nil {
return Function{}, fmt.Errorf("unable to get task status for container: %s %s", name, err)
}
if svc.Status == "running" {
replicas = 1
f.pid = task.Pid()
// Get container IP address
ip, err := cninetwork.GetIPfromPID(int(task.Pid()))
if err != nil {
return Function{}, err
}
f.IP = ip.String()
}
} else {
replicas = 0
}
f.replicas = replicas
return f, nil
}
return Function{}, fmt.Errorf("unable to find function: %s, error %s", name, err)
}

View File

@ -12,27 +12,27 @@ const (
OrchestrationIdentifier = "containerd"
// ProviderName name of the provider
ProviderName = "faasd-ce"
ProviderName = "faasd"
)
// MakeInfoHandler creates handler for /system/info endpoint
//MakeInfoHandler creates handler for /system/info endpoint
func MakeInfoHandler(version, sha string) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if r.Body != nil {
defer r.Body.Close()
}
infoResponse := types.ProviderInfo{
infoResponse := types.InfoResponse{
Orchestration: OrchestrationIdentifier,
Name: ProviderName,
Version: &types.VersionInfo{
Provider: ProviderName,
Version: types.ProviderVersion{
Release: version,
SHA: sha,
},
}
jsonOut, err := json.Marshal(infoResponse)
if err != nil {
jsonOut, marshalErr := json.Marshal(infoResponse)
if marshalErr != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
@ -42,6 +42,3 @@ func MakeInfoHandler(version, sha string) http.HandlerFunc {
w.Write(jsonOut)
}
}
const faasdMaxFunctions = 15
const faasdMaxNs = 1

View File

@ -15,14 +15,14 @@ func Test_InfoHandler(t *testing.T) {
r := httptest.NewRequest("GET", "/", nil)
handler(w, r)
resp := types.ProviderInfo{}
resp := types.InfoResponse{}
err := json.Unmarshal(w.Body.Bytes(), &resp)
if err != nil {
t.Fatalf("unexpected error unmarshalling the response")
}
if resp.Name != ProviderName {
t.Fatalf("expected provider %q, got %q", ProviderName, resp.Name)
if resp.Provider != ProviderName {
t.Fatalf("expected provider %q, got %q", ProviderName, resp.Provider)
}
if resp.Orchestration != OrchestrationIdentifier {

View File

@ -4,10 +4,8 @@ import (
"fmt"
"log"
"net/url"
"strings"
"github.com/containerd/containerd"
faasd "github.com/openfaas/faasd/pkg"
)
const watchdogPort = 8080
@ -21,18 +19,11 @@ func NewInvokeResolver(client *containerd.Client) *InvokeResolver {
}
func (i *InvokeResolver) Resolve(functionName string) (url.URL, error) {
actualFunctionName := functionName
log.Printf("Resolve: %q\n", actualFunctionName)
log.Printf("Resolve: %q\n", functionName)
namespace := getNamespaceOrDefault(functionName, faasd.DefaultFunctionNamespace)
if strings.Contains(functionName, ".") {
actualFunctionName = strings.TrimSuffix(functionName, "."+namespace)
}
function, err := GetFunction(i.client, actualFunctionName, namespace)
function, err := GetFunction(i.client, functionName)
if err != nil {
return url.URL{}, fmt.Errorf("%s not found", actualFunctionName)
return url.URL{}, fmt.Errorf("%s not found", functionName)
}
serviceIP := function.IP
@ -46,11 +37,3 @@ func (i *InvokeResolver) Resolve(functionName string) (url.URL, error) {
return *urlRes, nil
}
func getNamespaceOrDefault(name, defaultNamespace string) string {
namespace := defaultNamespace
if strings.Contains(name, ".") {
namespace = name[strings.LastIndexAny(name, ".")+1:]
}
return namespace
}

View File

@ -1,285 +0,0 @@
package handlers
import (
"context"
"encoding/json"
"fmt"
"io"
"log"
"net/http"
"strings"
"github.com/containerd/containerd"
"github.com/gorilla/mux"
"github.com/openfaas/faas-provider/types"
)
func MakeMutateNamespace(client *containerd.Client) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
if r.Body != nil {
defer r.Body.Close()
}
switch r.Method {
case http.MethodPost:
createNamespace(client, w, r)
case http.MethodGet:
getNamespace(client, w, r)
case http.MethodDelete:
deleteNamespace(client, w, r)
case http.MethodPut:
updateNamespace(client, w, r)
default:
w.WriteHeader(http.StatusMethodNotAllowed)
}
}
}
func updateNamespace(client *containerd.Client, w http.ResponseWriter, r *http.Request) {
req, err := parseNamespaceRequest(r)
if err != nil {
http.Error(w, err.Error(), err.(*HttpError).Status)
return
}
namespaceExists, err := namespaceExists(r.Context(), client, req.Name)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !namespaceExists {
http.Error(w, fmt.Sprintf("namespace %s not found", req.Name), http.StatusNotFound)
return
}
originalLabels, err := client.NamespaceService().Labels(r.Context(), req.Name)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !hasOpenFaaSLabel(originalLabels) {
http.Error(w, fmt.Sprintf("namespace %s is not an openfaas namespace", req.Name), http.StatusBadRequest)
return
}
var exclusions []string
// build exclusions
for key, _ := range originalLabels {
if _, ok := req.Labels[key]; !ok {
exclusions = append(exclusions, key)
}
}
// Call SetLabel with empty string if label is to be removed
for _, key := range exclusions {
if err := client.NamespaceService().SetLabel(r.Context(), req.Name, key, ""); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
}
// Now add the new labels
for key, value := range req.Labels {
if err := client.NamespaceService().SetLabel(r.Context(), req.Name, key, value); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
}
w.WriteHeader(http.StatusAccepted)
}
func deleteNamespace(client *containerd.Client, w http.ResponseWriter, r *http.Request) {
req, err := parseNamespaceRequest(r)
if err != nil {
http.Error(w, err.Error(), err.(*HttpError).Status)
return
}
if err := client.NamespaceService().Delete(r.Context(), req.Name); err != nil {
if strings.Contains(err.Error(), "not found") {
http.Error(w, fmt.Sprintf("namespace %s not found", req.Name), http.StatusNotFound)
return
}
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusAccepted)
}
func namespaceExists(ctx context.Context, client *containerd.Client, name string) (bool, error) {
ns, err := client.NamespaceService().List(ctx)
if err != nil {
return false, err
}
found := false
for _, namespace := range ns {
if namespace == name {
found = true
break
}
}
return found, nil
}
func getNamespace(client *containerd.Client, w http.ResponseWriter, r *http.Request) {
req, err := parseNamespaceRequest(r)
if err != nil {
http.Error(w, err.Error(), err.(*HttpError).Status)
return
}
namespaceExists, err := namespaceExists(r.Context(), client, req.Name)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !namespaceExists {
http.Error(w, fmt.Sprintf("namespace %s not found", req.Name), http.StatusNotFound)
return
}
labels, err := client.NamespaceService().Labels(r.Context(), req.Name)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !hasOpenFaaSLabel(labels) {
http.Error(w, fmt.Sprintf("namespace %s not found", req.Name), http.StatusNotFound)
return
}
res := types.FunctionNamespace{
Name: req.Name,
Labels: labels,
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(res); err != nil {
log.Printf("Get Namespace error: %s", err)
}
}
func createNamespace(client *containerd.Client, w http.ResponseWriter, r *http.Request) {
req, err := parseNamespaceRequest(r)
if err != nil {
http.Error(w, err.Error(), err.(*HttpError).Status)
return
}
// Check if namespace exists, and it has the openfaas label
namespaces, err := client.NamespaceService().List(r.Context())
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
found := false
for _, namespace := range namespaces {
if namespace == req.Name {
found = true
break
}
}
if found {
http.Error(w, fmt.Sprintf("namespace %s already exists", req.Name), http.StatusConflict)
return
}
if err := client.NamespaceService().Create(r.Context(), req.Name, req.Labels); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
w.WriteHeader(http.StatusCreated)
}
// getNamespace returns a namespace object or an error
func parseNamespaceRequest(r *http.Request) (types.FunctionNamespace, error) {
var req types.FunctionNamespace
vars := mux.Vars(r)
namespaceInPath := vars["name"]
if r.Method == http.MethodGet {
if namespaceInPath == "" {
return req, &HttpError{
Err: fmt.Errorf("namespace not specified in URL"),
Status: http.StatusBadRequest,
}
}
return types.FunctionNamespace{
Name: namespaceInPath,
}, nil
}
body, _ := io.ReadAll(r.Body)
if err := json.Unmarshal(body, &req); err != nil {
return req, &HttpError{
Err: fmt.Errorf("error parsing request body: %s", err.Error()),
Status: http.StatusBadRequest,
}
}
if r.Method != http.MethodPost {
if namespaceInPath == "" {
return req, &HttpError{
Err: fmt.Errorf("namespace not specified in URL"),
Status: http.StatusBadRequest,
}
}
if req.Name != namespaceInPath {
return req, &HttpError{
Err: fmt.Errorf("namespace in request body does not match namespace in URL"),
Status: http.StatusBadRequest,
}
}
}
if req.Name == "" {
return req, &HttpError{
Err: fmt.Errorf("namespace not specified in request body"),
Status: http.StatusBadRequest,
}
}
if ok := hasOpenFaaSLabel(req.Labels); !ok {
return req, &HttpError{
Err: fmt.Errorf("request does not have openfaas=1 label"),
Status: http.StatusBadRequest,
}
}
return req, nil
}
func hasOpenFaaSLabel(labels map[string]string) bool {
if v, ok := labels["openfaas"]; ok && v == "1" {
return true
}
return false
}
type HttpError struct {
Err error
Status int
}
func (e *HttpError) Error() string {
return e.Err.Error()
}

View File

@ -1,62 +0,0 @@
package handlers
import (
"context"
"encoding/json"
"log"
"net/http"
"github.com/containerd/containerd"
"github.com/openfaas/faasd/pkg"
faasd "github.com/openfaas/faasd/pkg"
)
func MakeNamespacesLister(client *containerd.Client) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
list := ListNamespaces(client)
body, _ := json.Marshal(list)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
w.Write(body)
}
}
func ListNamespaces(client *containerd.Client) []string {
set := []string{faasd.DefaultFunctionNamespace}
store := client.NamespaceService()
namespaces, err := store.List(context.Background())
if err != nil {
log.Printf("Error listing namespaces: %s", err.Error())
return set
}
for _, namespace := range namespaces {
labels, err := store.Labels(context.Background(), namespace)
if err != nil {
log.Printf("Error listing label for namespace %s: %s", namespace, err.Error())
continue
}
if _, found := labels[pkg.NamespaceLabel]; found {
set = append(set, namespace)
}
}
if len(set) == 0 {
set = append(set, faasd.DefaultFunctionNamespace)
}
return set
}
func findNamespace(target string, items []string) bool {
for _, n := range items {
if n == target {
return true
}
}
return false
}

View File

@ -5,8 +5,6 @@ import (
"log"
"net/http"
"k8s.io/apimachinery/pkg/api/resource"
"github.com/containerd/containerd"
"github.com/openfaas/faas-provider/types"
)
@ -15,57 +13,27 @@ func MakeReadHandler(client *containerd.Client) func(w http.ResponseWriter, r *h
return func(w http.ResponseWriter, r *http.Request) {
lookupNamespace := getRequestNamespace(readNamespaceFromQuery(r))
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client.NamespaceService(), lookupNamespace)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
res := []types.FunctionStatus{}
fns, err := ListFunctions(client, lookupNamespace)
funcs, err := ListFunctions(client)
if err != nil {
log.Printf("[Read] error listing functions. Error: %s", err)
log.Printf("[Read] error listing functions. Error: %s\n", err)
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
for _, fn := range fns {
annotations := &fn.annotations
labels := &fn.labels
memory := resource.NewQuantity(fn.memoryLimit, resource.BinarySI)
status := types.FunctionStatus{
Name: fn.name,
Image: fn.image,
Replicas: uint64(fn.replicas),
Namespace: fn.namespace,
Labels: labels,
Annotations: annotations,
Secrets: fn.secrets,
EnvVars: fn.envVars,
EnvProcess: fn.envProcess,
CreatedAt: fn.createdAt,
}
// Do not remove below memory check for 0
// Memory limit should not be included in status until set explicitly
limit := &types.FunctionResources{Memory: memory.String()}
if limit.Memory != "0" {
status.Limits = limit
}
res = append(res, status)
for _, function := range funcs {
res = append(res, types.FunctionStatus{
Name: function.name,
Image: function.image,
Replicas: uint64(function.replicas),
Namespace: function.namespace,
Labels: &function.labels,
})
}
body, _ := json.Marshal(res)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
w.Write(body)
}
}

View File

@ -14,33 +14,14 @@ func MakeReplicaReaderHandler(client *containerd.Client) func(w http.ResponseWri
return func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
functionName := vars["name"]
lookupNamespace := getRequestNamespace(readNamespaceFromQuery(r))
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client.NamespaceService(), lookupNamespace)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
if f, err := GetFunction(client, functionName, lookupNamespace); err == nil {
if f, err := GetFunction(client, functionName); err == nil {
found := types.FunctionStatus{
Name: functionName,
Image: f.image,
AvailableReplicas: uint64(f.replicas),
Replicas: uint64(f.replicas),
Namespace: f.namespace,
Labels: &f.labels,
Annotations: &f.annotations,
Secrets: f.secrets,
EnvVars: f.envVars,
EnvProcess: f.envProcess,
CreatedAt: f.createdAt,
}
functionBytes, _ := json.Marshal(found)

View File

@ -4,16 +4,15 @@ import (
"context"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"github.com/containerd/containerd"
"github.com/containerd/containerd/namespaces"
gocni "github.com/containerd/go-cni"
"github.com/openfaas/faas-provider/types"
"github.com/openfaas/faasd/pkg"
faasd "github.com/openfaas/faasd/pkg"
)
func MakeReplicaUpdateHandler(client *containerd.Client, cni gocni.CNI) func(w http.ResponseWriter, r *http.Request) {
@ -27,43 +26,29 @@ func MakeReplicaUpdateHandler(client *containerd.Client, cni gocni.CNI) func(w h
defer r.Body.Close()
body, _ := io.ReadAll(r.Body)
body, _ := ioutil.ReadAll(r.Body)
log.Printf("[Scale] request: %s\n", string(body))
req := types.ScaleServiceRequest{}
if err := json.Unmarshal(body, &req); err != nil {
log.Printf("[Scale] error parsing input: %s", err)
http.Error(w, err.Error(), http.StatusBadRequest)
err := json.Unmarshal(body, &req)
return
}
namespace := req.Namespace
if namespace == "" {
namespace = pkg.DefaultFunctionNamespace
}
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client.NamespaceService(), namespace)
if err != nil {
log.Printf("[Scale] error parsing input: %s\n", err)
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
name := req.ServiceName
if _, err := GetFunction(client, name, namespace); err != nil {
msg := fmt.Sprintf("function: %s.%s not found", name, namespace)
if _, err := GetFunction(client, name); err != nil {
msg := fmt.Sprintf("service %s not found", name)
log.Printf("[Scale] %s\n", msg)
http.Error(w, msg, http.StatusNotFound)
return
}
ctx := namespaces.WithNamespace(context.Background(), namespace)
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace)
ctr, ctrErr := client.LoadContainer(ctx, name)
if ctrErr != nil {
@ -73,63 +58,46 @@ func MakeReplicaUpdateHandler(client *containerd.Client, cni gocni.CNI) func(w h
return
}
var taskExists bool
var taskStatus *containerd.Status
taskExists := true
task, taskErr := ctr.Task(ctx, nil)
if taskErr != nil {
msg := fmt.Sprintf("cannot load task for service %s, error: %s", name, taskErr)
log.Printf("[Scale] %s\n", msg)
taskExists = false
} else {
taskExists = true
status, statusErr := task.Status(ctx)
if statusErr != nil {
msg := fmt.Sprintf("cannot load task status for %s, error: %s", name, statusErr)
log.Printf("[Scale] %s\n", msg)
http.Error(w, msg, http.StatusInternalServerError)
return
}
if req.Replicas > 0 {
if taskExists {
if status, statusErr := task.Status(ctx); statusErr == nil {
if status.Status == containerd.Paused {
if resumeErr := task.Resume(ctx); resumeErr != nil {
log.Printf("[Scale] error resuming task %s, error: %s\n", name, resumeErr)
http.Error(w, resumeErr.Error(), http.StatusBadRequest)
}
}
}
} else {
taskStatus = &status
deployErr := createTask(ctx, client, ctr, cni)
if deployErr != nil {
log.Printf("[Scale] error deploying %s, error: %s\n", name, deployErr)
http.Error(w, deployErr.Error(), http.StatusBadRequest)
return
}
return
}
}
createNewTask := false
if req.Replicas == 0 {
http.Error(w, "replicas must > 0 for faasd CE", http.StatusBadRequest)
return
}
if taskExists {
if taskStatus != nil {
if taskStatus.Status == containerd.Paused {
if _, err := task.Delete(ctx); err != nil {
log.Printf("[Scale] error deleting paused task %s, error: %s\n", name, err)
http.Error(w, err.Error(), http.StatusBadRequest)
return
} else {
if taskExists {
if status, statusErr := task.Status(ctx); statusErr == nil {
if status.Status == containerd.Running {
if pauseErr := task.Pause(ctx); pauseErr != nil {
log.Printf("[Scale] error pausing task %s, error: %s\n", name, pauseErr)
http.Error(w, pauseErr.Error(), http.StatusBadRequest)
}
}
} else if taskStatus.Status == containerd.Stopped {
// Stopped tasks cannot be restarted, must be removed, and created again
if _, err := task.Delete(ctx); err != nil {
log.Printf("[Scale] error deleting stopped task %s, error: %s\n", name, err)
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
createNewTask = true
}
}
} else {
createNewTask = true
}
if createNewTask {
deployErr := createTask(ctx, ctr, cni)
if deployErr != nil {
log.Printf("[Scale] error deploying %s, error: %s\n", name, deployErr)
http.Error(w, deployErr.Error(), http.StatusBadRequest)
return
}
}
}
}

View File

@ -3,21 +3,20 @@ package handlers
import (
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"path"
"strings"
"github.com/containerd/containerd"
"github.com/openfaas/faas-provider/types"
provider "github.com/openfaas/faasd/pkg/provider"
)
const secretFilePermission = 0644
const secretDirPermission = 0755
func MakeSecretHandler(store provider.Labeller, mountPath string) func(w http.ResponseWriter, r *http.Request) {
func MakeSecretHandler(c *containerd.Client, mountPath string) func(w http.ResponseWriter, r *http.Request) {
err := os.MkdirAll(mountPath, secretFilePermission)
if err != nil {
@ -31,13 +30,13 @@ func MakeSecretHandler(store provider.Labeller, mountPath string) func(w http.Re
switch r.Method {
case http.MethodGet:
listSecrets(store, w, r, mountPath)
listSecrets(c, w, r, mountPath)
case http.MethodPost:
createSecret(w, r, mountPath)
createSecret(c, w, r, mountPath)
case http.MethodPut:
createSecret(w, r, mountPath)
createSecret(c, w, r, mountPath)
case http.MethodDelete:
deleteSecret(w, r, mountPath)
deleteSecret(c, w, r, mountPath)
default:
w.WriteHeader(http.StatusBadRequest)
return
@ -46,46 +45,23 @@ func MakeSecretHandler(store provider.Labeller, mountPath string) func(w http.Re
}
}
func listSecrets(store provider.Labeller, w http.ResponseWriter, r *http.Request, mountPath string) {
lookupNamespace := getRequestNamespace(readNamespaceFromQuery(r))
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(store, lookupNamespace)
func listSecrets(c *containerd.Client, w http.ResponseWriter, r *http.Request, mountPath string) {
files, err := ioutil.ReadDir(mountPath)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
mountPath = getNamespaceSecretMountPath(mountPath, lookupNamespace)
files, err := os.ReadDir(mountPath)
if os.IsNotExist(err) {
bytesOut, _ := json.Marshal([]types.Secret{})
w.Write(bytesOut)
return
}
if err != nil {
log.Printf("[Secret] Error listing secrets: %s ", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
secrets := []types.Secret{}
for _, f := range files {
secrets = append(secrets, types.Secret{Name: f.Name(), Namespace: lookupNamespace})
secrets = append(secrets, types.Secret{Name: f.Name()})
}
bytesOut, _ := json.Marshal(secrets)
w.Write(bytesOut)
}
func createSecret(w http.ResponseWriter, r *http.Request, mountPath string) {
func createSecret(c *containerd.Client, w http.ResponseWriter, r *http.Request, mountPath string) {
secret, err := parseSecret(r)
if err != nil {
log.Printf("[secret] error %s", err.Error())
@ -93,30 +69,7 @@ func createSecret(w http.ResponseWriter, r *http.Request, mountPath string) {
return
}
err = validateSecret(secret)
if err != nil {
log.Printf("[secret] error %s", err.Error())
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
log.Printf("[secret] is valid: %q", secret.Name)
namespace := getRequestNamespace(secret.Namespace)
mountPath = getNamespaceSecretMountPath(mountPath, namespace)
err = os.MkdirAll(mountPath, secretDirPermission)
if err != nil {
log.Printf("[secret] error %s", err.Error())
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
data := secret.RawValue
if len(data) == 0 {
data = []byte(secret.Value)
}
err = os.WriteFile(path.Join(mountPath, secret.Name), data, secretFilePermission)
err = ioutil.WriteFile(path.Join(mountPath, secret.Name), []byte(secret.Value), secretFilePermission)
if err != nil {
log.Printf("[secret] error %s", err.Error())
@ -125,7 +78,7 @@ func createSecret(w http.ResponseWriter, r *http.Request, mountPath string) {
}
}
func deleteSecret(w http.ResponseWriter, r *http.Request, mountPath string) {
func deleteSecret(c *containerd.Client, w http.ResponseWriter, r *http.Request, mountPath string) {
secret, err := parseSecret(r)
if err != nil {
log.Printf("[secret] error %s", err.Error())
@ -133,9 +86,6 @@ func deleteSecret(w http.ResponseWriter, r *http.Request, mountPath string) {
return
}
namespace := getRequestNamespace(readNamespaceFromQuery(r))
mountPath = getNamespaceSecretMountPath(mountPath, namespace)
err = os.Remove(path.Join(mountPath, secret.Name))
if err != nil {
@ -147,7 +97,7 @@ func deleteSecret(w http.ResponseWriter, r *http.Request, mountPath string) {
func parseSecret(r *http.Request) (types.Secret, error) {
secret := types.Secret{}
bytesOut, err := io.ReadAll(r.Body)
bytesOut, err := ioutil.ReadAll(r.Body)
if err != nil {
return secret, err
}
@ -157,6 +107,10 @@ func parseSecret(r *http.Request) (types.Secret, error) {
return secret, err
}
if isTraversal(secret.Name) {
return secret, fmt.Errorf(traverseErrorSt)
}
return secret, err
}
@ -166,13 +120,3 @@ func isTraversal(name string) bool {
return strings.Contains(name, fmt.Sprintf("%s", string(os.PathSeparator))) ||
strings.Contains(name, "..")
}
func validateSecret(secret types.Secret) error {
if strings.TrimSpace(secret.Name) == "" {
return fmt.Errorf("non-empty name is required")
}
if isTraversal(secret.Name) {
return fmt.Errorf(traverseErrorSt)
}
return nil
}

View File

@ -1,252 +1,63 @@
package handlers
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"github.com/openfaas/faas-provider/types"
"github.com/openfaas/faasd/pkg"
provider "github.com/openfaas/faasd/pkg/provider"
)
func Test_parseSecret(t *testing.T) {
cases := []struct {
name string
payload string
expError string
expSecret types.Secret
}{
{
name: "no error when name is valid without extention and with no traversal",
payload: `{"name": "authorized_keys", "value": "foo"}`,
expSecret: types.Secret{Name: "authorized_keys", Value: "foo"},
},
{
name: "no error when name is valid and parses RawValue correctly",
payload: `{"name": "authorized_keys", "rawValue": "YmFy"}`,
expSecret: types.Secret{Name: "authorized_keys", RawValue: []byte("bar")},
},
{
name: "no error when name is valid with dot and with no traversal",
payload: `{"name": "authorized.keys", "value": "foo"}`,
expSecret: types.Secret{Name: "authorized.keys", Value: "foo"},
},
}
func Test_parseSecretValidName(t *testing.T) {
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
reader := strings.NewReader(tc.payload)
r := httptest.NewRequest(http.MethodPost, "/", reader)
secret, err := parseSecret(r)
if err != nil && tc.expError == "" {
t.Fatalf("unexpected error: %s", err)
return
}
s := types.Secret{Name: "authorized_keys"}
body, _ := json.Marshal(s)
reader := bytes.NewReader(body)
r := httptest.NewRequest(http.MethodPost, "/", reader)
_, err := parseSecret(r)
if tc.expError != "" {
if err == nil {
t.Fatalf("expected error: %s, got nil", tc.expError)
}
if err.Error() != tc.expError {
t.Fatalf("expected error: %s, got: %s", tc.expError, err)
}
return
}
if !reflect.DeepEqual(secret, tc.expSecret) {
t.Fatalf("expected secret: %+v, got: %+v", tc.expSecret, secret)
}
})
}
}
func TestSecretCreation(t *testing.T) {
mountPath, err := os.MkdirTemp("", "test_secret_creation")
if err != nil {
t.Fatalf("unexpected error while creating temp directory: %s", err)
}
defer os.RemoveAll(mountPath)
handler := MakeSecretHandler(nil, mountPath)
cases := []struct {
name string
verb string
payload string
status int
secretPath string
secret string
err string
}{
{
name: "returns error when the name contains a traversal",
verb: http.MethodPost,
payload: `{"name": "/root/.ssh/authorized_keys", "value": "foo"}`,
status: http.StatusBadRequest,
err: "directory traversal found in name\n",
},
{
name: "returns error when the name contains a traversal",
verb: http.MethodPost,
payload: `{"name": "..", "value": "foo"}`,
status: http.StatusBadRequest,
err: "directory traversal found in name\n",
},
{
name: "empty request returns a validation error",
verb: http.MethodPost,
payload: `{}`,
status: http.StatusBadRequest,
err: "non-empty name is required\n",
},
{
name: "can create secret from string",
verb: http.MethodPost,
payload: `{"name": "foo", "value": "bar"}`,
status: http.StatusOK,
secretPath: "/openfaas-fn/foo",
secret: "bar",
},
{
name: "can create secret from raw value",
verb: http.MethodPost,
payload: `{"name": "foo", "rawValue": "YmFy"}`,
status: http.StatusOK,
secretPath: "/openfaas-fn/foo",
secret: "bar",
},
{
name: "can create secret in non-default namespace from raw value",
verb: http.MethodPost,
payload: `{"name": "pity", "rawValue": "dGhlIGZvbw==", "namespace": "a-team"}`,
status: http.StatusOK,
secretPath: "/a-team/pity",
secret: "the foo",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
req := httptest.NewRequest(tc.verb, "http://example.com/foo", strings.NewReader(tc.payload))
w := httptest.NewRecorder()
handler(w, req)
resp := w.Result()
if resp.StatusCode != tc.status {
t.Logf("response body: %s", w.Body.String())
t.Fatalf("expected status: %d, got: %d", tc.status, resp.StatusCode)
}
if resp.StatusCode != http.StatusOK && w.Body.String() != tc.err {
t.Fatalf("expected error message: %q, got %q", tc.err, w.Body.String())
}
if tc.secretPath != "" {
data, err := os.ReadFile(filepath.Join(mountPath, tc.secretPath))
if err != nil {
t.Fatalf("can not read the secret from disk: %s", err)
}
if string(data) != tc.secret {
t.Fatalf("expected secret value: %s, got %s", tc.secret, string(data))
}
}
})
t.Fatalf("secret name is valid with no traversal characters")
}
}
func TestListSecrets(t *testing.T) {
mountPath, err := os.MkdirTemp("", "test_secret_creation")
func Test_parseSecretValidNameWithDot(t *testing.T) {
s := types.Secret{Name: "authorized.keys"}
body, _ := json.Marshal(s)
reader := bytes.NewReader(body)
r := httptest.NewRequest(http.MethodPost, "/", reader)
_, err := parseSecret(r)
if err != nil {
t.Fatalf("unexpected error while creating temp directory: %s", err)
}
defer os.RemoveAll(mountPath)
cases := []struct {
name string
verb string
namespace string
labels map[string]string
status int
secretPath string
secret string
err string
expected []types.Secret
}{
{
name: "Get empty secret list for default namespace having no secret",
verb: http.MethodGet,
status: http.StatusOK,
secretPath: "/test-fn/foo",
secret: "bar",
expected: make([]types.Secret, 0),
},
{
name: "Get empty secret list for non-default namespace having no secret",
verb: http.MethodGet,
status: http.StatusOK,
secretPath: "/test-fn/foo",
secret: "bar",
expected: make([]types.Secret, 0),
namespace: "other-ns",
labels: map[string]string{
pkg.NamespaceLabel: "true",
},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
labelStore := provider.NewFakeLabeller(tc.labels)
handler := MakeSecretHandler(labelStore, mountPath)
path := "http://example.com/foo"
if len(tc.namespace) > 0 {
path = path + fmt.Sprintf("?namespace=%s", tc.namespace)
}
req := httptest.NewRequest(tc.verb, path, nil)
w := httptest.NewRecorder()
handler(w, req)
resp := w.Result()
if resp.StatusCode != tc.status {
t.Fatalf("want status: %d, but got: %d", tc.status, resp.StatusCode)
}
if resp.StatusCode != http.StatusOK && w.Body.String() != tc.err {
t.Fatalf("want error message: %q, but got %q", tc.err, w.Body.String())
}
body, err := io.ReadAll(resp.Body)
if err != nil {
t.Fatalf("can't read response of list %v", err)
}
var res []types.Secret
err = json.Unmarshal(body, &res)
if err != nil {
t.Fatalf("unable to unmarshal %q, error: %v", string(body), err)
}
if !reflect.DeepEqual(res, tc.expected) {
t.Fatalf("want response: %v, but got: %v", tc.expected, res)
}
})
t.Fatalf("secret name is valid with no traversal characters")
}
}
func Test_parseSecretWithTraversalWithSlash(t *testing.T) {
s := types.Secret{Name: "/root/.ssh/authorized_keys"}
body, _ := json.Marshal(s)
reader := bytes.NewReader(body)
r := httptest.NewRequest(http.MethodPost, "/", reader)
_, err := parseSecret(r)
if err == nil {
t.Fatalf("secret name should fail due to path traversal")
}
}
func Test_parseSecretWithTraversalWithDoubleDot(t *testing.T) {
s := types.Secret{Name: ".."}
body, _ := json.Marshal(s)
reader := bytes.NewReader(body)
r := httptest.NewRequest(http.MethodPost, "/", reader)
_, err := parseSecret(r)
if err == nil {
t.Fatalf("secret name should fail due to path traversal")
}
}

View File

@ -4,7 +4,7 @@ import (
"context"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
@ -13,6 +13,7 @@ import (
gocni "github.com/containerd/go-cni"
"github.com/openfaas/faas-provider/types"
faasd "github.com/openfaas/faasd/pkg"
"github.com/openfaas/faasd/pkg/cninetwork"
"github.com/openfaas/faasd/pkg/service"
)
@ -28,60 +29,33 @@ func MakeUpdateHandler(client *containerd.Client, cni gocni.CNI, secretMountPath
defer r.Body.Close()
body, _ := io.ReadAll(r.Body)
body, _ := ioutil.ReadAll(r.Body)
log.Printf("[Update] request: %s\n", string(body))
req := types.FunctionDeployment{}
err := json.Unmarshal(body, &req)
if err != nil {
log.Printf("[Update] error parsing input: %s", err)
log.Printf("[Update] error parsing input: %s\n", err)
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
name := req.Service
namespace := getRequestNamespace(req.Namespace)
// Check if namespace exists, and it has the openfaas label
valid, err := validNamespace(client.NamespaceService(), namespace)
function, err := GetFunction(client, name)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if !valid {
http.Error(w, "namespace not valid", http.StatusBadRequest)
return
}
if err := preDeploy(client, int64(0)); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
log.Printf("[Deploy] error deploying %s, error: %s\n", name, err)
return
}
namespaceSecretMountPath := getNamespaceSecretMountPath(secretMountPath, namespace)
function, err := GetFunction(client, name, namespace)
if err != nil {
msg := fmt.Sprintf("function: %s.%s not found", name, namespace)
msg := fmt.Sprintf("service %s not found", name)
log.Printf("[Update] %s\n", msg)
http.Error(w, msg, http.StatusNotFound)
return
}
err = validateSecrets(namespaceSecretMountPath, req.Secrets)
err = validateSecrets(secretMountPath, req.Secrets)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
}
ctx := namespaces.WithNamespace(context.Background(), namespace)
if _, err := prepull(ctx, req, client, alwaysPull); err != nil {
log.Printf("[Update] error with pre-pull: %s, %s\n", name, err)
http.Error(w, err.Error(), http.StatusInternalServerError)
}
ctx := namespaces.WithNamespace(context.Background(), faasd.FunctionNamespace)
if function.replicas != 0 {
err = cninetwork.DeleteCNINetwork(ctx, cni, client, name)
if err != nil {
@ -89,19 +63,19 @@ func MakeUpdateHandler(client *containerd.Client, cni gocni.CNI, secretMountPath
}
}
if err := service.Remove(ctx, client, name); err != nil {
log.Printf("[Update] error removing %s, %s\n", name, err)
http.Error(w, err.Error(), http.StatusInternalServerError)
containerErr := service.Remove(ctx, client, name)
if containerErr != nil {
log.Printf("[Update] error removing %s, %s\n", name, containerErr)
http.Error(w, containerErr.Error(), http.StatusInternalServerError)
return
}
// The pull has already been done in prepull, so we can force this pull to "false"
pull := false
if err := deploy(ctx, req, client, cni, namespaceSecretMountPath, pull); err != nil {
log.Printf("[Update] error deploying %s, error: %s\n", name, err)
http.Error(w, err.Error(), http.StatusBadRequest)
deployErr := deploy(ctx, req, client, cni, secretMountPath, alwaysPull)
if deployErr != nil {
log.Printf("[Update] error deploying %s, error: %s\n", name, deployErr)
http.Error(w, deployErr.Error(), http.StatusBadRequest)
return
}
}
}

View File

@ -1,48 +0,0 @@
package handlers
import (
"context"
"net/http"
"path"
"github.com/openfaas/faasd/pkg"
faasd "github.com/openfaas/faasd/pkg"
provider "github.com/openfaas/faasd/pkg/provider"
)
func getRequestNamespace(namespace string) string {
if len(namespace) > 0 {
return namespace
}
return faasd.DefaultFunctionNamespace
}
func readNamespaceFromQuery(r *http.Request) string {
q := r.URL.Query()
return q.Get("namespace")
}
func getNamespaceSecretMountPath(userSecretPath string, namespace string) string {
return path.Join(userSecretPath, namespace)
}
// validNamespace indicates whether the namespace is eligable to be
// used for OpenFaaS functions.
func validNamespace(store provider.Labeller, namespace string) (bool, error) {
if namespace == faasd.DefaultFunctionNamespace {
return true, nil
}
labels, err := store.Labels(context.Background(), namespace)
if err != nil {
return false, err
}
// check for true to keep it backward compatible
if value, found := labels[pkg.NamespaceLabel]; found && (value == "true" || value == "1") {
return true, nil
}
return false, nil
}

View File

@ -1,75 +0,0 @@
package handlers
import (
"fmt"
"net/http"
"net/http/httptest"
"testing"
faasd "github.com/openfaas/faasd/pkg"
)
func Test_getRequestNamespace(t *testing.T) {
tables := []struct {
name string
requestNamespace string
expectedNamespace string
}{
{name: "RequestNamespace is not provided", requestNamespace: "", expectedNamespace: faasd.DefaultFunctionNamespace},
{name: "RequestNamespace is provided", requestNamespace: "user-namespace", expectedNamespace: "user-namespace"},
}
for _, tc := range tables {
t.Run(tc.name, func(t *testing.T) {
actualNamespace := getRequestNamespace(tc.requestNamespace)
if actualNamespace != tc.expectedNamespace {
t.Errorf("Want: %s, got: %s", actualNamespace, tc.expectedNamespace)
}
})
}
}
func Test_getNamespaceSecretMountPath(t *testing.T) {
userSecretPath := "/var/openfaas/secrets"
tables := []struct {
name string
requestNamespace string
expectedSecretPath string
}{
{name: "Default Namespace is provided", requestNamespace: faasd.DefaultFunctionNamespace, expectedSecretPath: "/var/openfaas/secrets/" + faasd.DefaultFunctionNamespace},
{name: "User Namespace is provided", requestNamespace: "user-namespace", expectedSecretPath: "/var/openfaas/secrets/user-namespace"},
}
for _, tc := range tables {
t.Run(tc.name, func(t *testing.T) {
actualNamespace := getNamespaceSecretMountPath(userSecretPath, tc.requestNamespace)
if actualNamespace != tc.expectedSecretPath {
t.Errorf("Want: %s, got: %s", actualNamespace, tc.expectedSecretPath)
}
})
}
}
func Test_readNamespaceFromQuery(t *testing.T) {
tables := []struct {
name string
queryNamespace string
expectedNamespace string
}{
{name: "No Namespace is provided", queryNamespace: "", expectedNamespace: ""},
{name: "User Namespace is provided", queryNamespace: "user-namespace", expectedNamespace: "user-namespace"},
}
for _, tc := range tables {
t.Run(tc.name, func(t *testing.T) {
url := fmt.Sprintf("/test?namespace=%s", tc.queryNamespace)
r := httptest.NewRequest(http.MethodGet, url, nil)
actualNamespace := readNamespaceFromQuery(r)
if actualNamespace != tc.expectedNamespace {
t.Errorf("Want: %s, got: %s", actualNamespace, tc.expectedNamespace)
}
})
}
}

View File

@ -1,25 +0,0 @@
package provider
import "context"
// Labeller can return labels for a namespace from containerd.
type Labeller interface {
Labels(ctx context.Context, namespace string) (map[string]string, error)
}
//
// FakeLabeller can be used to fake labels applied on namespace to mark
// them valid/invalid for openfaas functions
type FakeLabeller struct {
labels map[string]string
}
func NewFakeLabeller(labels map[string]string) Labeller {
return &FakeLabeller{
labels: labels,
}
}
func (s *FakeLabeller) Labels(ctx context.Context, namespace string) (map[string]string, error) {
return s.labels, nil
}

View File

@ -6,91 +6,74 @@ import (
"log"
"net"
"net/http"
"strconv"
"strings"
"time"
)
// NewProxy creates a HTTP proxy to expose a host
func NewProxy(upstream string, listenPort uint32, hostIP string, timeout time.Duration, resolver Resolver) *Proxy {
// NewProxy creates a HTTP proxy to expose the gateway container
// from OpenFaaS to the host
func NewProxy(port int, timeout time.Duration) *Proxy {
return &Proxy{
Upstream: upstream,
Port: listenPort,
HostIP: hostIP,
Timeout: timeout,
Resolver: resolver,
Port: port,
Timeout: timeout,
}
}
// Proxy for exposing a private container
type Proxy struct {
Timeout time.Duration
Port int
}
// Port on which to listen to traffic
Port uint32
// Upstream is where to send traffic when received
Upstream string
// The IP to use to bind locally
HostIP string
Resolver Resolver
type proxyState struct {
Host string
}
// Start listening and forwarding HTTP to the host
func (p *Proxy) Start() error {
func (p *Proxy) Start(gatewayChan chan string, done chan bool) error {
tcp := p.Port
http.DefaultClient.CheckRedirect = func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
}
upstreamHost, upstreamPort, err := getUpstream(p.Upstream, p.Port)
if err != nil {
return err
ps := proxyState{
Host: "",
}
log.Printf("Looking up IP for: %q", upstreamHost)
got := make(chan string, 1)
ps.Host = <-gatewayChan
go p.Resolver.Get(upstreamHost, got, time.Second*5)
log.Printf("Starting faasd proxy on %d\n", tcp)
ipAddress := <-got
close(got)
fmt.Printf("Gateway: %s\n", ps.Host)
upstreamAddr := fmt.Sprintf("%s:%d", ipAddress, upstreamPort)
localBind := fmt.Sprintf("%s:%d", p.HostIP, p.Port)
log.Printf("Proxy from: %s, to: %s (%s)\n", localBind, p.Upstream, ipAddress)
l, err := net.Listen("tcp", localBind)
l, err := net.Listen("tcp", fmt.Sprintf("0.0.0.0:%d", tcp))
if err != nil {
log.Printf("Error: %s", err.Error())
return err
}
defer l.Close()
for {
// Wait for a connection.
conn, err := l.Accept()
if err != nil {
log.Printf("Unable to accept on: %d, error: %s",
p.Port,
err.Error())
return err
acceptErr := fmt.Errorf("Unable to accept on %d, error: %s", tcp, err.Error())
log.Printf("%s", acceptErr.Error())
return acceptErr
}
upstream, err := net.Dial("tcp", upstreamAddr)
if err != nil {
conn.Close()
upstream, err := net.Dial("tcp", fmt.Sprintf("%s", ps.Host))
log.Printf("Unable to dial: %s, error: %s", upstreamAddr, err.Error())
if err != nil {
log.Printf("unable to dial to %s, error: %s", ps.Host, err.Error())
return err
}
go pipe(conn, upstream)
go pipe(upstream, conn)
}
}
@ -98,19 +81,3 @@ func pipe(from net.Conn, to net.Conn) {
defer from.Close()
io.Copy(from, to)
}
func getUpstream(val string, defaultPort uint32) (string, uint32, error) {
upstreamHostname := val
upstreamPort := defaultPort
if in := strings.Index(val, ":"); in > -1 {
upstreamHostname = val[:in]
port, err := strconv.ParseInt(val[in+1:], 10, 32)
if err != nil {
return "", defaultPort, err
}
upstreamPort = uint32(port)
}
return upstreamHostname, upstreamPort, nil
}

View File

@ -2,7 +2,7 @@ package pkg
import (
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"net/http/httptest"
@ -16,7 +16,7 @@ func Test_Proxy_ToPrivateServer(t *testing.T) {
wantBodyText := "OK"
wantBody := []byte(wantBodyText)
upstreamSvr := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Body != nil {
defer r.Body.Close()
@ -27,19 +27,17 @@ func Test_Proxy_ToPrivateServer(t *testing.T) {
}))
defer upstreamSvr.Close()
defer upstream.Close()
port := 8080
u, _ := url.Parse(upstreamSvr.URL)
log.Println("Host", u.Host)
upstreamAddr := u.Host
proxy := NewProxy(upstreamAddr, 8080, "127.0.0.1", time.Second*1, &mockResolver{})
proxy := NewProxy(port, time.Second*1)
gwChan := make(chan string, 1)
doneCh := make(chan bool)
go proxy.Start()
go proxy.Start(gwChan, doneCh)
u, _ := url.Parse(upstream.URL)
log.Println("Host", u.Host)
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
@ -61,7 +59,7 @@ func Test_Proxy_ToPrivateServer(t *testing.T) {
time.Sleep(time.Millisecond * 100)
} else {
resBody, _ := io.ReadAll(res.Body)
resBody, _ := ioutil.ReadAll(res.Body)
if string(resBody) != string(wantBody) {
t.Errorf("want %s, but got %s in body", string(wantBody), string(resBody))
}
@ -73,14 +71,3 @@ func Test_Proxy_ToPrivateServer(t *testing.T) {
doneCh <- true
}()
}
type mockResolver struct {
}
func (m *mockResolver) Start() {
}
func (m *mockResolver) Get(upstream string, got chan<- string, timeout time.Duration) {
got <- upstream
}

View File

@ -1,12 +0,0 @@
package pkg
import "time"
// Resolver resolves an upstream IP address for a given upstream host
type Resolver interface {
// Start any polling or connections required to resolve
Start()
// Get an IP address using an asynchronous operation
Get(upstream string, got chan<- string, timeout time.Duration)
}

View File

@ -6,7 +6,6 @@ import (
"log"
"os"
"path/filepath"
"strings"
"sync"
"time"
@ -28,51 +27,34 @@ func Remove(ctx context.Context, client *containerd.Client, name string) error {
container, containerErr := client.LoadContainer(ctx, name)
if containerErr == nil {
taskFound := true
found := true
t, err := container.Task(ctx, nil)
if err != nil {
if errdefs.IsNotFound(err) {
taskFound = false
found = false
} else {
return fmt.Errorf("unable to get task %w: ", err)
return fmt.Errorf("unable to get task %s: ", err)
}
}
if taskFound {
status, err := t.Status(ctx)
if found {
status, _ := t.Status(ctx)
fmt.Printf("Status of %s is: %s\n", name, status.Status)
log.Printf("Need to kill %s\n", name)
err := killTask(ctx, t)
if err != nil {
log.Printf("Unable to get status for: %s, error: %s", name, err.Error())
} else {
log.Printf("Status of %s is: %s\n", name, status.Status)
return fmt.Errorf("error killing task %s, %s, %s", container.ID(), name, err)
}
var gracePeriod = time.Second * 30
spec, err := t.Spec(ctx)
if err == nil {
for _, p := range spec.Process.Env {
k, v, ok := strings.Cut(p, "=")
if ok && k == "grace_period" {
periodVal, err := time.ParseDuration(v)
if err == nil {
gracePeriod = periodVal
}
}
}
}
if err = killTask(ctx, t, gracePeriod); err != nil {
return fmt.Errorf("error killing task %s, %s, %w", container.ID(), name, err)
}
}
if err := container.Delete(ctx, containerd.WithSnapshotCleanup); err != nil {
return fmt.Errorf("error deleting container %s, %s, %w", container.ID(), name, err)
err = container.Delete(ctx, containerd.WithSnapshotCleanup)
if err != nil {
return fmt.Errorf("error deleting container %s, %s, %s", container.ID(), name, err)
}
} else {
service := client.SnapshotService("")
key := name + "-snapshot"
key := name + "snapshot"
if _, err := client.SnapshotService("").Stat(ctx, key); err == nil {
service.Remove(ctx, key)
}
@ -81,61 +63,43 @@ func Remove(ctx context.Context, client *containerd.Client, name string) error {
}
// Adapted from Stellar - https://github.com/stellar
func killTask(ctx context.Context, task containerd.Task, gracePeriod time.Duration) error {
func killTask(ctx context.Context, task containerd.Task) error {
killTimeout := 30 * time.Second
wg := &sync.WaitGroup{}
wg.Add(1)
var err error
waited := false
go func() {
defer wg.Done()
if task != nil {
wait, err := task.Wait(ctx)
if err != nil {
log.Printf("error waiting on task: %s", err)
err = fmt.Errorf("error waiting on task: %s", err)
return
}
if err := task.Kill(ctx, unix.SIGTERM, containerd.WithKillAll); err != nil {
log.Printf("error killing container task: %s", err)
}
select {
case <-wait:
waited = true
task.Delete(ctx)
return
case <-time.After(gracePeriod):
log.Printf("Sending SIGKILL to: %s after: %s", task.ID(), gracePeriod.Round(time.Second).String())
case <-time.After(killTimeout):
if err := task.Kill(ctx, unix.SIGKILL, containerd.WithKillAll); err != nil {
log.Printf("error sending SIGKILL to task: %s", err)
log.Printf("error force killing container task: %s", err)
}
return
}
}
}()
wg.Wait()
if task != nil {
if !waited {
wait, err := task.Wait(ctx)
if err != nil {
log.Printf("error waiting on task after kill: %s", err)
}
<-wait
}
if _, err := task.Delete(ctx); err != nil {
return err
}
}
return err
}
func getResolver(configFile *configfile.ConfigFile) (remotes.Resolver, error) {
func getResolver(ctx context.Context, configFile *configfile.ConfigFile) (remotes.Resolver, error) {
// credsFunc is based on https://github.com/moby/buildkit/blob/0b130cca040246d2ddf55117eeff34f546417e40/session/auth/authprovider/authprovider.go#L35
credFunc := func(host string) (string, string, error) {
if host == "registry-1.docker.io" {
@ -150,7 +114,6 @@ func getResolver(configFile *configfile.ConfigFile) (remotes.Resolver, error) {
}
return ac.Username, ac.Password, nil
}
authOpts := []docker.AuthorizerOpt{docker.WithAuthCreds(credFunc)}
authorizer := docker.NewDockerAuthorizer(authOpts...)
opts := docker.ResolverOptions{
@ -165,17 +128,17 @@ func PrepareImage(ctx context.Context, client *containerd.Client, imageName, sna
resolver remotes.Resolver
)
if _, statErr := os.Stat(filepath.Join(dockerConfigDir, config.ConfigFileName)); statErr == nil {
if _, stErr := os.Stat(filepath.Join(dockerConfigDir, config.ConfigFileName)); stErr == nil {
configFile, err := config.Load(dockerConfigDir)
if err != nil {
return nil, err
}
resolver, err = getResolver(configFile)
resolver, err = getResolver(ctx, configFile)
if err != nil {
return empty, err
}
} else if !os.IsNotExist(statErr) {
return empty, statErr
} else if !os.IsNotExist(stErr) {
return empty, stErr
}
var image containerd.Image
@ -187,6 +150,7 @@ func PrepareImage(ctx context.Context, client *containerd.Client, imageName, sna
image = img
} else {
img, err := client.GetImage(ctx, imageName)
if err != nil {
if !errdefs.IsNotFound(err) {
@ -223,11 +187,9 @@ func pullImage(ctx context.Context, client *containerd.Client, resolver remotes.
rOpts := []containerd.RemoteOpt{
containerd.WithPullUnpack,
}
if resolver != nil {
rOpts = append(rOpts, containerd.WithResolver(resolver))
}
img, err := client.Pull(ctx, imageName, rOpts...)
if err != nil {
return empty, fmt.Errorf("cannot pull: %s", err)

View File

@ -3,14 +3,13 @@ package pkg
import (
"context"
"fmt"
"io/ioutil"
"log"
"os"
"path"
"sort"
"strconv"
"strings"
"github.com/alexellis/arkade/pkg/env"
"github.com/alexellis/k3sup/pkg/env"
"github.com/compose-spec/compose-go/loader"
compose "github.com/compose-spec/compose-go/types"
"github.com/containerd/containerd"
@ -18,23 +17,23 @@ import (
"github.com/containerd/containerd/containers"
"github.com/containerd/containerd/oci"
gocni "github.com/containerd/go-cni"
"github.com/distribution/reference"
"github.com/openfaas/faasd/pkg/cninetwork"
"github.com/openfaas/faasd/pkg/service"
"github.com/pkg/errors"
"github.com/containerd/containerd/namespaces"
units "github.com/docker/go-units"
"github.com/opencontainers/runtime-spec/specs-go"
)
const (
// workingDirectoryPermission user read/write/execute, group and others: read-only
workingDirectoryPermission = 0744
defaultSnapshotter = "overlayfs"
workingDirectoryPermission = 0644
// faasdNamespace is the containerd namespace services are created
faasdNamespace = "default"
faasServicesPullAlways = false
)
type Service struct {
// Image is the container image registry reference, in an OCI format.
Image string
Env []string
Name string
@ -42,28 +41,11 @@ type Service struct {
Caps []string
Args []string
DependsOn []string
Ports []ServicePort
// User in the docker-compose.yaml spec can set as follows:
// a user-id, username, userid:groupid or user:group
User string
}
type ServicePort struct {
TargetPort uint32
Port uint32
HostIP string
}
type Mount struct {
// Src relative to the working directory for faasd
Src string
// Dest is the absolute path within the container
Src string
Dest string
// ReadOnly when set to true indicates the mount will be set to "ro" instead of "rw"
ReadOnly bool
}
type Supervisor struct {
@ -89,7 +71,7 @@ func NewSupervisor(sock string) (*Supervisor, error) {
}
func (s *Supervisor) Start(svcs []Service) error {
ctx := namespaces.WithNamespace(context.Background(), FaasdNamespace)
ctx := namespaces.WithNamespace(context.Background(), faasdNamespace)
wd, _ := os.Getwd()
@ -101,7 +83,7 @@ func (s *Supervisor) Start(svcs []Service) error {
127.0.0.1 localhost
%s faasd-provider`, gw)
writeHostsErr := os.WriteFile(path.Join(wd, "hosts"),
writeHostsErr := ioutil.WriteFile(path.Join(wd, "hosts"),
[]byte(hosts), workingDirectoryPermission)
if writeHostsErr != nil {
@ -113,20 +95,13 @@ func (s *Supervisor) Start(svcs []Service) error {
for _, svc := range svcs {
fmt.Printf("Preparing %s with image: %s\n", svc.Name, svc.Image)
r, err := reference.ParseNormalizedNamed(svc.Image)
if err != nil {
return err
}
imgRef := reference.TagNameOnly(r).String()
img, err := service.PrepareImage(ctx, s.client, imgRef, defaultSnapshotter, faasServicesPullAlways)
img, err := service.PrepareImage(ctx, s.client, svc.Image, defaultSnapshotter, faasServicesPullAlways)
if err != nil {
return err
}
images[svc.Name] = img
size, _ := img.Size(ctx)
fmt.Printf("Prepare done for: %s, %s\n", svc.Image, units.HumanSize(float64(size)))
fmt.Printf("Prepare done for: %s, %d bytes\n", svc.Image, size)
}
for _, svc := range svcs {
@ -156,41 +131,12 @@ func (s *Supervisor) Start(svcs []Service) error {
mounts := []specs.Mount{}
if len(svc.Mounts) > 0 {
for _, mnt := range svc.Mounts {
var options = []string{"rbind"}
if mnt.ReadOnly {
options = append(options, "ro")
} else {
options = append(options, "rw")
}
mounts = append(mounts, specs.Mount{
Source: mnt.Src,
Destination: mnt.Dest,
Type: "bind",
Options: options,
Options: []string{"rbind", "rw"},
})
// Only create directories, not files.
// Some files don't have a suffix, such as secrets.
if len(path.Ext(mnt.Src)) == 0 &&
!strings.HasPrefix(mnt.Src, "/var/lib/faasd/secrets/") {
// src is already prefixed with wd from an earlier step
src := mnt.Src
fmt.Printf("Creating local directory: %s\n", src)
if err := os.MkdirAll(src, workingDirectoryPermission); err != nil {
if !errors.Is(os.ErrExist, err) {
fmt.Printf("Unable to create: %s, %s\n", src, err)
}
}
if len(svc.User) > 0 {
uid, err := strconv.Atoi(svc.User)
if err == nil {
if err := os.Chown(src, uid, -1); err != nil {
fmt.Printf("Unable to chown: %s to %d, error: %s\n", src, uid, err)
}
}
}
}
}
}
@ -208,67 +154,60 @@ func (s *Supervisor) Start(svcs []Service) error {
Options: []string{"rbind", "ro"},
})
if len(svc.User) > 0 {
log.Printf("Running %s with user: %q", svc.Name, svc.User)
}
newContainer, err := s.client.NewContainer(
newContainer, containerCreateErr := s.client.NewContainer(
ctx,
svc.Name,
containerd.WithImage(image),
containerd.WithNewSnapshot(svc.Name+"-snapshot", image),
containerd.WithNewSpec(oci.WithImageConfig(image),
oci.WithHostname(svc.Name),
withUserOrDefault(svc.User),
oci.WithCapabilities(svc.Caps),
oci.WithMounts(mounts),
withOCIArgs(svc.Args),
oci.WithEnv(svc.Env)),
)
if err != nil {
log.Printf("Error creating container: %s", err)
return err
if containerCreateErr != nil {
log.Printf("Error creating container: %s\n", containerCreateErr)
return containerCreateErr
}
log.Printf("Created container: %s\n", newContainer.ID())
task, err := newContainer.NewTask(ctx, cio.BinaryIO("/usr/local/bin/faasd", nil))
task, err := newContainer.NewTask(ctx, cio.NewCreator(cio.WithStdio))
if err != nil {
log.Printf("Error creating task: %s", err)
log.Printf("Error creating task: %s\n", err)
return err
}
labels := map[string]string{}
_, err = cninetwork.CreateCNINetwork(ctx, s.cni, task, labels)
network, err := cninetwork.CreateCNINetwork(ctx, s.cni, task, labels)
if err != nil {
log.Printf("Error creating CNI for %s: %s", svc.Name, err)
return err
}
ip, err := cninetwork.GetIPAddress(svc.Name, task.Pid())
ip, err := cninetwork.GetIPAddress(network, task)
if err != nil {
log.Printf("Error getting IP for %s: %s", svc.Name, err)
return err
}
log.Printf("%s has IP: %s\n", newContainer.ID(), ip)
log.Printf("%s has IP: %s\n", newContainer.ID(), ip.String())
hosts, err := os.ReadFile("hosts")
if err != nil {
log.Printf("Unable to read hosts file: %s\n", err.Error())
}
hosts, _ := ioutil.ReadFile("hosts")
hosts = []byte(string(hosts) + fmt.Sprintf(`
%s %s
`, ip, svc.Name))
writeErr := ioutil.WriteFile("hosts", hosts, workingDirectoryPermission)
if err := os.WriteFile("hosts", hosts, workingDirectoryPermission); err != nil {
log.Printf("Error writing file: %s %s\n", "hosts", err)
if writeErr != nil {
log.Printf("Error writing file %s %s\n", "hosts", writeErr)
}
// os.Chown("hosts", 101, 101)
if _, err := task.Wait(ctx); err != nil {
log.Printf("Task wait error: %s", err)
_, err = task.Wait(ctx)
if err != nil {
log.Printf("Wait err: %s\n", err)
return err
}
@ -276,7 +215,7 @@ func (s *Supervisor) Start(svcs []Service) error {
// log.Println("Exited: ", exitStatusC)
if err = task.Start(ctx); err != nil {
log.Printf("Task start error: %s", err)
log.Printf("Task err: %s\n", err)
return err
}
}
@ -289,7 +228,7 @@ func (s *Supervisor) Close() {
}
func (s *Supervisor) Remove(svcs []Service) error {
ctx := namespaces.WithNamespace(context.Background(), FaasdNamespace)
ctx := namespaces.WithNamespace(context.Background(), faasdNamespace)
for _, svc := range svcs {
err := cninetwork.DeleteCNINetwork(ctx, s.cni, s.client, svc.Name)
@ -306,16 +245,6 @@ func (s *Supervisor) Remove(svcs []Service) error {
return nil
}
func withUserOrDefault(userstr string) oci.SpecOpts {
if len(userstr) > 0 {
return oci.WithUser(userstr)
}
return func(_ context.Context, _ oci.Client, _ *containers.Container, s *oci.Spec) error {
return nil
}
}
func withOCIArgs(args []string) oci.SpecOpts {
if len(args) > 0 {
return oci.WithProcessArgs(args...)
@ -354,9 +283,8 @@ func ParseCompose(config *compose.Config) ([]Service, error) {
return nil, errors.Errorf("unsupported volume mount type '%s' when parsing service '%s'", v.Type, s.Name)
}
mounts = append(mounts, Mount{
Src: v.Source,
Dest: v.Target,
ReadOnly: v.ReadOnly,
Src: v.Source,
Dest: v.Target,
})
}
@ -369,27 +297,12 @@ func ParseCompose(config *compose.Config) ([]Service, error) {
Env: env,
Mounts: mounts,
DependsOn: s.DependsOn,
User: s.User,
Ports: convertPorts(s.Ports),
}
}
return services, nil
}
func convertPorts(ports []compose.ServicePortConfig) []ServicePort {
servicePorts := []ServicePort{}
for _, p := range ports {
servicePorts = append(servicePorts, ServicePort{
Port: p.Published,
TargetPort: p.Target,
HostIP: p.HostIP,
})
}
return servicePorts
}
// LoadComposeFile is a helper method for loading a docker-compose file
func LoadComposeFile(wd string, file string) (*compose.Config, error) {
return LoadComposeFileWithArch(wd, file, env.GetClientArch)
@ -399,7 +312,7 @@ func LoadComposeFile(wd string, file string) (*compose.Config, error) {
func LoadComposeFileWithArch(wd string, file string, archGetter ArchGetter) (*compose.Config, error) {
file = path.Join(wd, file)
b, err := os.ReadFile(file)
b, err := ioutil.ReadFile(file)
if err != nil {
return nil, err
}
@ -451,6 +364,8 @@ func GetArchSuffix(getClientArch ArchGetter) (suffix string, err error) {
case "x86_64":
// no suffix needed
return "", nil
case "armhf", "armv7l":
return "-armhf", nil
case "arm64", "aarch64":
return "-arm64", nil
default:

View File

@ -180,7 +180,7 @@ func equalMountSlice(t *testing.T, want, found []Mount) {
for i := range want {
if !reflect.DeepEqual(want[i], found[i]) {
t.Fatalf("unexpected value at postition %d: want %v, got %v", i, want[i], found[i])
t.Fatalf("unexpected value at postition %d: want %s, got %s", i, want[i], found[i])
}
}
}
@ -210,6 +210,18 @@ func Test_GetArchSuffix(t *testing.T) {
foundArch: "anything_else",
want: "",
},
{
name: "armhf has armhf suffix",
foundOS: "Linux",
foundArch: "armhf",
want: "-armhf",
},
{
name: "armv7l has armhf suffix",
foundOS: "Linux",
foundArch: "armv7l",
want: "-armhf",
},
{
name: "arm64 has arm64 suffix",
foundOS: "Linux",

View File

@ -2,23 +2,21 @@ package systemd
import (
"bytes"
"context"
"fmt"
"os"
"path/filepath"
"text/template"
execute "github.com/alexellis/go-execute/v2"
execute "github.com/alexellis/go-execute/pkg/v1"
)
func Enable(unit string) error {
task := execute.ExecTask{
Command: "systemctl",
task := execute.ExecTask{Command: "systemctl",
Args: []string{"enable", unit},
StreamStdio: false,
}
res, err := task.Execute(context.Background())
res, err := task.Execute()
if err != nil {
return err
}
@ -31,13 +29,12 @@ func Enable(unit string) error {
}
func Start(unit string) error {
task := execute.ExecTask{
Command: "systemctl",
task := execute.ExecTask{Command: "systemctl",
Args: []string{"start", unit},
StreamStdio: false,
}
res, err := task.Execute(context.Background())
res, err := task.Execute()
if err != nil {
return err
}
@ -50,13 +47,12 @@ func Start(unit string) error {
}
func DaemonReload() error {
task := execute.ExecTask{
Command: "systemctl",
task := execute.ExecTask{Command: "systemctl",
Args: []string{"daemon-reload"},
StreamStdio: false,
}
res, err := task.Execute(context.Background())
res, err := task.Execute()
if err != nil {
return err
}
@ -75,20 +71,23 @@ func InstallUnit(name string, tokens map[string]string) error {
tmplName := "./hack/" + name + ".service"
tmpl, err := template.ParseFiles(tmplName)
if err != nil {
return fmt.Errorf("error loading template %s, error %s", tmplName, err)
}
var tpl bytes.Buffer
if err := tmpl.Execute(&tpl, tokens); err != nil {
err = tmpl.Execute(&tpl, tokens)
if err != nil {
return err
}
if err := writeUnit(name+".service", tpl.Bytes()); err != nil {
err = writeUnit(name+".service", tpl.Bytes())
if err != nil {
return err
}
return nil
}
@ -97,12 +96,7 @@ func writeUnit(name string, data []byte) error {
if err != nil {
return err
}
defer f.Close()
if _, err := f.Write(data); err != nil {
return err
}
return nil
_, err = f.Write(data)
return err
}

View File

@ -26,8 +26,6 @@ services:
- "8222"
- "--store=memory"
- "--cluster_id=faas-cluster"
ports:
- "127.0.0.1:8222:8222"
prometheus:
image: docker.io/prom/prometheus:v2.14.0
@ -37,8 +35,6 @@ services:
target: /etc/prometheus/prometheus.yml
cap_add:
- CAP_NET_RAW
ports:
- "127.0.0.1:9090:9090"
gateway:
image: "docker.io/openfaas/gateway:0.18.17${ARCH_SUFFIX}"
@ -69,8 +65,6 @@ services:
- basic-auth-plugin
- nats
- prometheus
ports:
- "8080:8080"
queue-worker:
image: docker.io/openfaas/queue-worker:0.11.2

View File

@ -1,17 +1 @@
package pkg
// These values will be injected into these variables at the build time.
var (
// GitCommit Git Commit SHA
GitCommit string
// Version version of the CLI
Version string
)
// GetVersion get latest version
func GetVersion() string {
if len(Version) == 0 {
return "dev"
}
return Version
}

View File

@ -26,7 +26,3 @@ scrape_configs:
- job_name: 'gateway'
static_configs:
- targets: ['gateway:8082']
- job_name: 'provider'
static_configs:
- targets: ['faasd-provider:8081']

View File

@ -1,93 +0,0 @@
# go-fuzz-headers
This repository contains various helper functions for go fuzzing. It is mostly used in combination with [go-fuzz](https://github.com/dvyukov/go-fuzz), but compatibility with fuzzing in the standard library will also be supported. Any coverage guided fuzzing engine that provides an array or slice of bytes can be used with go-fuzz-headers.
## Usage
Using go-fuzz-headers is easy. First create a new consumer with the bytes provided by the fuzzing engine:
```go
import (
fuzz "github.com/AdaLogics/go-fuzz-headers"
)
data := []byte{'R', 'a', 'n', 'd', 'o', 'm'}
f := fuzz.NewConsumer(data)
```
This creates a `Consumer` that consumes the bytes of the input as it uses them to fuzz different types.
After that, `f` can be used to easily create fuzzed instances of different types. Below are some examples:
### Structs
One of the most useful features of go-fuzz-headers is its ability to fill structs with the data provided by the fuzzing engine. This is done with a single line:
```go
type Person struct {
Name string
Age int
}
p := Person{}
// Fill p with values based on the data provided by the fuzzing engine:
err := f.GenerateStruct(&p)
```
This includes nested structs too. In this example, the fuzz Consumer will also insert values in `p.BestFriend`:
```go
type PersonI struct {
Name string
Age int
BestFriend PersonII
}
type PersonII struct {
Name string
Age int
}
p := PersonI{}
err := f.GenerateStruct(&p)
```
If the consumer should insert values for unexported fields as well as exported, this can be enabled with:
```go
f.AllowUnexportedFields()
```
...and disabled with:
```go
f.DisallowUnexportedFields()
```
### Other types:
Other useful APIs:
```go
createdString, err := f.GetString() // Gets a string
createdInt, err := f.GetInt() // Gets an integer
createdByte, err := f.GetByte() // Gets a byte
createdBytes, err := f.GetBytes() // Gets a byte slice
createdBool, err := f.GetBool() // Gets a boolean
err := f.FuzzMap(target_map) // Fills a map
createdTarBytes, err := f.TarBytes() // Gets bytes of a valid tar archive
err := f.CreateFiles(inThisDir) // Fills inThisDir with files
createdString, err := f.GetStringFrom("anyCharInThisString", ofThisLength) // Gets a string that consists of chars from "anyCharInThisString" and has the exact length "ofThisLength"
```
Most APIs are added as they are needed.
## Projects that use go-fuzz-headers
- [runC](https://github.com/opencontainers/runc)
- [Istio](https://github.com/istio/istio)
- [Vitess](https://github.com/vitessio/vitess)
- [Containerd](https://github.com/containerd/containerd)
Feel free to add your own project to the list, if you use go-fuzz-headers to fuzz it.
## Status
The project is under development and will be updated regularly.
## References
go-fuzz-headers' approach to fuzzing structs is strongly inspired by [gofuzz](https://github.com/google/gofuzz).

View File

@ -1,914 +0,0 @@
// Copyright 2023 The go-fuzz-headers Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package gofuzzheaders
import (
"archive/tar"
"bytes"
"encoding/binary"
"errors"
"fmt"
"io"
"math"
"os"
"path/filepath"
"reflect"
"strconv"
"strings"
"time"
"unsafe"
)
var (
MaxTotalLen uint32 = 2000000
maxDepth = 100
)
func SetMaxTotalLen(newLen uint32) {
MaxTotalLen = newLen
}
type ConsumeFuzzer struct {
data []byte
dataTotal uint32
CommandPart []byte
RestOfArray []byte
NumberOfCalls int
position uint32
fuzzUnexportedFields bool
curDepth int
Funcs map[reflect.Type]reflect.Value
}
func IsDivisibleBy(n int, divisibleby int) bool {
return (n % divisibleby) == 0
}
func NewConsumer(fuzzData []byte) *ConsumeFuzzer {
return &ConsumeFuzzer{
data: fuzzData,
dataTotal: uint32(len(fuzzData)),
Funcs: make(map[reflect.Type]reflect.Value),
curDepth: 0,
}
}
func (f *ConsumeFuzzer) Split(minCalls, maxCalls int) error {
if f.dataTotal == 0 {
return errors.New("could not split")
}
numberOfCalls := int(f.data[0])
if numberOfCalls < minCalls || numberOfCalls > maxCalls {
return errors.New("bad number of calls")
}
if int(f.dataTotal) < numberOfCalls+numberOfCalls+1 {
return errors.New("length of data does not match required parameters")
}
// Define part 2 and 3 of the data array
commandPart := f.data[1 : numberOfCalls+1]
restOfArray := f.data[numberOfCalls+1:]
// Just a small check. It is necessary
if len(commandPart) != numberOfCalls {
return errors.New("length of commandPart does not match number of calls")
}
// Check if restOfArray is divisible by numberOfCalls
if !IsDivisibleBy(len(restOfArray), numberOfCalls) {
return errors.New("length of commandPart does not match number of calls")
}
f.CommandPart = commandPart
f.RestOfArray = restOfArray
f.NumberOfCalls = numberOfCalls
return nil
}
func (f *ConsumeFuzzer) AllowUnexportedFields() {
f.fuzzUnexportedFields = true
}
func (f *ConsumeFuzzer) DisallowUnexportedFields() {
f.fuzzUnexportedFields = false
}
func (f *ConsumeFuzzer) GenerateStruct(targetStruct interface{}) error {
e := reflect.ValueOf(targetStruct).Elem()
return f.fuzzStruct(e, false)
}
func (f *ConsumeFuzzer) setCustom(v reflect.Value) error {
// First: see if we have a fuzz function for it.
doCustom, ok := f.Funcs[v.Type()]
if !ok {
return fmt.Errorf("could not find a custom function")
}
switch v.Kind() {
case reflect.Ptr:
if v.IsNil() {
if !v.CanSet() {
return fmt.Errorf("could not use a custom function")
}
v.Set(reflect.New(v.Type().Elem()))
}
case reflect.Map:
if v.IsNil() {
if !v.CanSet() {
return fmt.Errorf("could not use a custom function")
}
v.Set(reflect.MakeMap(v.Type()))
}
default:
return fmt.Errorf("could not use a custom function")
}
verr := doCustom.Call([]reflect.Value{v, reflect.ValueOf(Continue{
F: f,
})})
// check if we return an error
if verr[0].IsNil() {
return nil
}
return fmt.Errorf("could not use a custom function")
}
func (f *ConsumeFuzzer) fuzzStruct(e reflect.Value, customFunctions bool) error {
if f.curDepth >= maxDepth {
// return err or nil here?
return nil
}
f.curDepth++
defer func() { f.curDepth-- }()
// We check if we should check for custom functions
if customFunctions && e.IsValid() && e.CanAddr() {
err := f.setCustom(e.Addr())
if err != nil {
return err
}
}
switch e.Kind() {
case reflect.Struct:
for i := 0; i < e.NumField(); i++ {
var v reflect.Value
if !e.Field(i).CanSet() {
if f.fuzzUnexportedFields {
v = reflect.NewAt(e.Field(i).Type(), unsafe.Pointer(e.Field(i).UnsafeAddr())).Elem()
}
if err := f.fuzzStruct(v, customFunctions); err != nil {
return err
}
} else {
v = e.Field(i)
if err := f.fuzzStruct(v, customFunctions); err != nil {
return err
}
}
}
case reflect.String:
str, err := f.GetString()
if err != nil {
return err
}
if e.CanSet() {
e.SetString(str)
}
case reflect.Slice:
var maxElements uint32
// Byte slices should not be restricted
if e.Type().String() == "[]uint8" {
maxElements = 10000000
} else {
maxElements = 50
}
randQty, err := f.GetUint32()
if err != nil {
return err
}
numOfElements := randQty % maxElements
if (f.dataTotal - f.position) < numOfElements {
numOfElements = f.dataTotal - f.position
}
uu := reflect.MakeSlice(e.Type(), int(numOfElements), int(numOfElements))
for i := 0; i < int(numOfElements); i++ {
// If we have more than 10, then we can proceed with that.
if err := f.fuzzStruct(uu.Index(i), customFunctions); err != nil {
if i >= 10 {
if e.CanSet() {
e.Set(uu)
}
return nil
} else {
return err
}
}
}
if e.CanSet() {
e.Set(uu)
}
case reflect.Uint16:
newInt, err := f.GetUint16()
if err != nil {
return err
}
if e.CanSet() {
e.SetUint(uint64(newInt))
}
case reflect.Uint32:
newInt, err := f.GetUint32()
if err != nil {
return err
}
if e.CanSet() {
e.SetUint(uint64(newInt))
}
case reflect.Uint64:
newInt, err := f.GetInt()
if err != nil {
return err
}
if e.CanSet() {
e.SetUint(uint64(newInt))
}
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
newInt, err := f.GetInt()
if err != nil {
return err
}
if e.CanSet() {
e.SetInt(int64(newInt))
}
case reflect.Float32:
newFloat, err := f.GetFloat32()
if err != nil {
return err
}
if e.CanSet() {
e.SetFloat(float64(newFloat))
}
case reflect.Float64:
newFloat, err := f.GetFloat64()
if err != nil {
return err
}
if e.CanSet() {
e.SetFloat(float64(newFloat))
}
case reflect.Map:
if e.CanSet() {
e.Set(reflect.MakeMap(e.Type()))
const maxElements = 50
randQty, err := f.GetInt()
if err != nil {
return err
}
numOfElements := randQty % maxElements
for i := 0; i < numOfElements; i++ {
key := reflect.New(e.Type().Key()).Elem()
if err := f.fuzzStruct(key, customFunctions); err != nil {
return err
}
val := reflect.New(e.Type().Elem()).Elem()
if err = f.fuzzStruct(val, customFunctions); err != nil {
return err
}
e.SetMapIndex(key, val)
}
}
case reflect.Ptr:
if e.CanSet() {
e.Set(reflect.New(e.Type().Elem()))
if err := f.fuzzStruct(e.Elem(), customFunctions); err != nil {
return err
}
return nil
}
case reflect.Uint8:
b, err := f.GetByte()
if err != nil {
return err
}
if e.CanSet() {
e.SetUint(uint64(b))
}
}
return nil
}
func (f *ConsumeFuzzer) GetStringArray() (reflect.Value, error) {
// The max size of the array:
const max uint32 = 20
arraySize := f.position
if arraySize > max {
arraySize = max
}
stringArray := reflect.MakeSlice(reflect.SliceOf(reflect.TypeOf("string")), int(arraySize), int(arraySize))
if f.position+arraySize >= f.dataTotal {
return stringArray, errors.New("could not make string array")
}
for i := 0; i < int(arraySize); i++ {
stringSize := uint32(f.data[f.position])
if f.position+stringSize >= f.dataTotal {
return stringArray, nil
}
stringToAppend := string(f.data[f.position : f.position+stringSize])
strVal := reflect.ValueOf(stringToAppend)
stringArray = reflect.Append(stringArray, strVal)
f.position += stringSize
}
return stringArray, nil
}
func (f *ConsumeFuzzer) GetInt() (int, error) {
if f.position >= f.dataTotal {
return 0, errors.New("not enough bytes to create int")
}
returnInt := int(f.data[f.position])
f.position++
return returnInt, nil
}
func (f *ConsumeFuzzer) GetByte() (byte, error) {
if f.position >= f.dataTotal {
return 0x00, errors.New("not enough bytes to get byte")
}
returnByte := f.data[f.position]
f.position++
return returnByte, nil
}
func (f *ConsumeFuzzer) GetNBytes(numberOfBytes int) ([]byte, error) {
if f.position >= f.dataTotal {
return nil, errors.New("not enough bytes to get byte")
}
returnBytes := make([]byte, 0, numberOfBytes)
for i := 0; i < numberOfBytes; i++ {
newByte, err := f.GetByte()
if err != nil {
return nil, err
}
returnBytes = append(returnBytes, newByte)
}
return returnBytes, nil
}
func (f *ConsumeFuzzer) GetUint16() (uint16, error) {
u16, err := f.GetNBytes(2)
if err != nil {
return 0, err
}
littleEndian, err := f.GetBool()
if err != nil {
return 0, err
}
if littleEndian {
return binary.LittleEndian.Uint16(u16), nil
}
return binary.BigEndian.Uint16(u16), nil
}
func (f *ConsumeFuzzer) GetUint32() (uint32, error) {
u32, err := f.GetNBytes(4)
if err != nil {
return 0, err
}
return binary.BigEndian.Uint32(u32), nil
}
func (f *ConsumeFuzzer) GetUint64() (uint64, error) {
u64, err := f.GetNBytes(8)
if err != nil {
return 0, err
}
littleEndian, err := f.GetBool()
if err != nil {
return 0, err
}
if littleEndian {
return binary.LittleEndian.Uint64(u64), nil
}
return binary.BigEndian.Uint64(u64), nil
}
func (f *ConsumeFuzzer) GetBytes() ([]byte, error) {
var length uint32
var err error
length, err = f.GetUint32()
if err != nil {
return nil, errors.New("not enough bytes to create byte array")
}
if length == 0 {
length = 30
}
bytesLeft := f.dataTotal - f.position
if bytesLeft <= 0 {
return nil, errors.New("not enough bytes to create byte array")
}
// If the length is the same as bytes left, we will not overflow
// the remaining bytes.
if length != bytesLeft {
length = length % bytesLeft
}
byteBegin := f.position
if byteBegin+length < byteBegin {
return nil, errors.New("numbers overflow")
}
f.position = byteBegin + length
return f.data[byteBegin:f.position], nil
}
func (f *ConsumeFuzzer) GetString() (string, error) {
if f.position >= f.dataTotal {
return "nil", errors.New("not enough bytes to create string")
}
length, err := f.GetUint32()
if err != nil {
return "nil", errors.New("not enough bytes to create string")
}
if f.position > MaxTotalLen {
return "nil", errors.New("created too large a string")
}
byteBegin := f.position
if byteBegin >= f.dataTotal {
return "nil", errors.New("not enough bytes to create string")
}
if byteBegin+length > f.dataTotal {
return "nil", errors.New("not enough bytes to create string")
}
if byteBegin > byteBegin+length {
return "nil", errors.New("numbers overflow")
}
f.position = byteBegin + length
return string(f.data[byteBegin:f.position]), nil
}
func (f *ConsumeFuzzer) GetBool() (bool, error) {
if f.position >= f.dataTotal {
return false, errors.New("not enough bytes to create bool")
}
if IsDivisibleBy(int(f.data[f.position]), 2) {
f.position++
return true, nil
} else {
f.position++
return false, nil
}
}
func (f *ConsumeFuzzer) FuzzMap(m interface{}) error {
return f.GenerateStruct(m)
}
func returnTarBytes(buf []byte) ([]byte, error) {
return buf, nil
// Count files
var fileCounter int
tr := tar.NewReader(bytes.NewReader(buf))
for {
_, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
fileCounter++
}
if fileCounter >= 1 {
return buf, nil
}
return nil, fmt.Errorf("not enough files were created\n")
}
func setTarHeaderFormat(hdr *tar.Header, f *ConsumeFuzzer) error {
ind, err := f.GetInt()
if err != nil {
hdr.Format = tar.FormatGNU
//return nil
}
switch ind % 4 {
case 0:
hdr.Format = tar.FormatUnknown
case 1:
hdr.Format = tar.FormatUSTAR
case 2:
hdr.Format = tar.FormatPAX
case 3:
hdr.Format = tar.FormatGNU
}
return nil
}
func setTarHeaderTypeflag(hdr *tar.Header, f *ConsumeFuzzer) error {
ind, err := f.GetInt()
if err != nil {
return err
}
switch ind % 13 {
case 0:
hdr.Typeflag = tar.TypeReg
case 1:
hdr.Typeflag = tar.TypeLink
linkname, err := f.GetString()
if err != nil {
return err
}
hdr.Linkname = linkname
case 2:
hdr.Typeflag = tar.TypeSymlink
linkname, err := f.GetString()
if err != nil {
return err
}
hdr.Linkname = linkname
case 3:
hdr.Typeflag = tar.TypeChar
case 4:
hdr.Typeflag = tar.TypeBlock
case 5:
hdr.Typeflag = tar.TypeDir
case 6:
hdr.Typeflag = tar.TypeFifo
case 7:
hdr.Typeflag = tar.TypeCont
case 8:
hdr.Typeflag = tar.TypeXHeader
case 9:
hdr.Typeflag = tar.TypeXGlobalHeader
case 10:
hdr.Typeflag = tar.TypeGNUSparse
case 11:
hdr.Typeflag = tar.TypeGNULongName
case 12:
hdr.Typeflag = tar.TypeGNULongLink
}
return nil
}
func (f *ConsumeFuzzer) createTarFileBody() ([]byte, error) {
return f.GetBytes()
/*length, err := f.GetUint32()
if err != nil {
return nil, errors.New("not enough bytes to create byte array")
}
// A bit of optimization to attempt to create a file body
// when we don't have as many bytes left as "length"
remainingBytes := f.dataTotal - f.position
if remainingBytes <= 0 {
return nil, errors.New("created too large a string")
}
if f.position+length > MaxTotalLen {
return nil, errors.New("created too large a string")
}
byteBegin := f.position
if byteBegin >= f.dataTotal {
return nil, errors.New("not enough bytes to create byte array")
}
if length == 0 {
return nil, errors.New("zero-length is not supported")
}
if byteBegin+length >= f.dataTotal {
return nil, errors.New("not enough bytes to create byte array")
}
if byteBegin+length < byteBegin {
return nil, errors.New("numbers overflow")
}
f.position = byteBegin + length
return f.data[byteBegin:f.position], nil*/
}
// getTarFileName is similar to GetString(), but creates string based
// on the length of f.data to reduce the likelihood of overflowing
// f.data.
func (f *ConsumeFuzzer) getTarFilename() (string, error) {
return f.GetString()
/*length, err := f.GetUint32()
if err != nil {
return "nil", errors.New("not enough bytes to create string")
}
// A bit of optimization to attempt to create a file name
// when we don't have as many bytes left as "length"
remainingBytes := f.dataTotal - f.position
if remainingBytes <= 0 {
return "nil", errors.New("created too large a string")
}
if f.position > MaxTotalLen {
return "nil", errors.New("created too large a string")
}
byteBegin := f.position
if byteBegin >= f.dataTotal {
return "nil", errors.New("not enough bytes to create string")
}
if byteBegin+length > f.dataTotal {
return "nil", errors.New("not enough bytes to create string")
}
if byteBegin > byteBegin+length {
return "nil", errors.New("numbers overflow")
}
f.position = byteBegin + length
return string(f.data[byteBegin:f.position]), nil*/
}
type TarFile struct {
Hdr *tar.Header
Body []byte
}
// TarBytes returns valid bytes for a tar archive
func (f *ConsumeFuzzer) TarBytes() ([]byte, error) {
numberOfFiles, err := f.GetInt()
if err != nil {
return nil, err
}
var tarFiles []*TarFile
tarFiles = make([]*TarFile, 0)
const maxNoOfFiles = 100
for i := 0; i < numberOfFiles%maxNoOfFiles; i++ {
var filename string
var filebody []byte
var sec, nsec int
var err error
filename, err = f.getTarFilename()
if err != nil {
var sb strings.Builder
sb.WriteString("file-")
sb.WriteString(strconv.Itoa(i))
filename = sb.String()
}
filebody, err = f.createTarFileBody()
if err != nil {
var sb strings.Builder
sb.WriteString("filebody-")
sb.WriteString(strconv.Itoa(i))
filebody = []byte(sb.String())
}
sec, err = f.GetInt()
if err != nil {
sec = 1672531200 // beginning of 2023
}
nsec, err = f.GetInt()
if err != nil {
nsec = 1703980800 // end of 2023
}
hdr := &tar.Header{
Name: filename,
Size: int64(len(filebody)),
Mode: 0o600,
ModTime: time.Unix(int64(sec), int64(nsec)),
}
if err := setTarHeaderTypeflag(hdr, f); err != nil {
return []byte(""), err
}
if err := setTarHeaderFormat(hdr, f); err != nil {
return []byte(""), err
}
tf := &TarFile{
Hdr: hdr,
Body: filebody,
}
tarFiles = append(tarFiles, tf)
}
var buf bytes.Buffer
tw := tar.NewWriter(&buf)
defer tw.Close()
for _, tf := range tarFiles {
tw.WriteHeader(tf.Hdr)
tw.Write(tf.Body)
}
return buf.Bytes(), nil
}
// This is similar to TarBytes, but it returns a series of
// files instead of raw tar bytes. The advantage of this
// api is that it is cheaper in terms of cpu power to
// modify or check the files in the fuzzer with TarFiles()
// because it avoids creating a tar reader.
func (f *ConsumeFuzzer) TarFiles() ([]*TarFile, error) {
numberOfFiles, err := f.GetInt()
if err != nil {
return nil, err
}
var tarFiles []*TarFile
tarFiles = make([]*TarFile, 0)
const maxNoOfFiles = 100
for i := 0; i < numberOfFiles%maxNoOfFiles; i++ {
filename, err := f.getTarFilename()
if err != nil {
return tarFiles, err
}
filebody, err := f.createTarFileBody()
if err != nil {
return tarFiles, err
}
sec, err := f.GetInt()
if err != nil {
return tarFiles, err
}
nsec, err := f.GetInt()
if err != nil {
return tarFiles, err
}
hdr := &tar.Header{
Name: filename,
Size: int64(len(filebody)),
Mode: 0o600,
ModTime: time.Unix(int64(sec), int64(nsec)),
}
if err := setTarHeaderTypeflag(hdr, f); err != nil {
hdr.Typeflag = tar.TypeReg
}
if err := setTarHeaderFormat(hdr, f); err != nil {
return tarFiles, err // should not happend
}
tf := &TarFile{
Hdr: hdr,
Body: filebody,
}
tarFiles = append(tarFiles, tf)
}
return tarFiles, nil
}
// CreateFiles creates pseudo-random files in rootDir.
// It creates subdirs and places the files there.
// It is the callers responsibility to ensure that
// rootDir exists.
func (f *ConsumeFuzzer) CreateFiles(rootDir string) error {
numberOfFiles, err := f.GetInt()
if err != nil {
return err
}
maxNumberOfFiles := numberOfFiles % 4000 // This is completely arbitrary
if maxNumberOfFiles == 0 {
return errors.New("maxNumberOfFiles is nil")
}
var noOfCreatedFiles int
for i := 0; i < maxNumberOfFiles; i++ {
// The file to create:
fileName, err := f.GetString()
if err != nil {
if noOfCreatedFiles > 0 {
// If files have been created, we don't return an error.
break
} else {
return errors.New("could not get fileName")
}
}
if strings.Contains(fileName, "..") || (len(fileName) > 0 && fileName[0] == 47) || strings.Contains(fileName, "\\") {
continue
}
fullFilePath := filepath.Join(rootDir, fileName)
// Find the subdirectory of the file
if subDir := filepath.Dir(fileName); subDir != "" && subDir != "." {
// create the dir first; avoid going outside the root dir
if strings.Contains(subDir, "../") || (len(subDir) > 0 && subDir[0] == 47) || strings.Contains(subDir, "\\") {
continue
}
dirPath := filepath.Join(rootDir, subDir)
if _, err := os.Stat(dirPath); os.IsNotExist(err) {
err2 := os.MkdirAll(dirPath, 0o777)
if err2 != nil {
continue
}
}
fullFilePath = filepath.Join(dirPath, fileName)
} else {
// Create symlink
createSymlink, err := f.GetBool()
if err != nil {
if noOfCreatedFiles > 0 {
break
} else {
return errors.New("could not create the symlink")
}
}
if createSymlink {
symlinkTarget, err := f.GetString()
if err != nil {
return err
}
err = os.Symlink(symlinkTarget, fullFilePath)
if err != nil {
return err
}
// stop loop here, since a symlink needs no further action
noOfCreatedFiles++
continue
}
// We create a normal file
fileContents, err := f.GetBytes()
if err != nil {
if noOfCreatedFiles > 0 {
break
} else {
return errors.New("could not create the file")
}
}
err = os.WriteFile(fullFilePath, fileContents, 0o666)
if err != nil {
continue
}
noOfCreatedFiles++
}
}
return nil
}
// GetStringFrom returns a string that can only consist of characters
// included in possibleChars. It returns an error if the created string
// does not have the specified length.
func (f *ConsumeFuzzer) GetStringFrom(possibleChars string, length int) (string, error) {
if (f.dataTotal - f.position) < uint32(length) {
return "", errors.New("not enough bytes to create a string")
}
output := make([]byte, 0, length)
for i := 0; i < length; i++ {
charIndex, err := f.GetInt()
if err != nil {
return string(output), err
}
output = append(output, possibleChars[charIndex%len(possibleChars)])
}
return string(output), nil
}
func (f *ConsumeFuzzer) GetRune() ([]rune, error) {
stringToConvert, err := f.GetString()
if err != nil {
return []rune("nil"), err
}
return []rune(stringToConvert), nil
}
func (f *ConsumeFuzzer) GetFloat32() (float32, error) {
u32, err := f.GetNBytes(4)
if err != nil {
return 0, err
}
littleEndian, err := f.GetBool()
if err != nil {
return 0, err
}
if littleEndian {
u32LE := binary.LittleEndian.Uint32(u32)
return math.Float32frombits(u32LE), nil
}
u32BE := binary.BigEndian.Uint32(u32)
return math.Float32frombits(u32BE), nil
}
func (f *ConsumeFuzzer) GetFloat64() (float64, error) {
u64, err := f.GetNBytes(8)
if err != nil {
return 0, err
}
littleEndian, err := f.GetBool()
if err != nil {
return 0, err
}
if littleEndian {
u64LE := binary.LittleEndian.Uint64(u64)
return math.Float64frombits(u64LE), nil
}
u64BE := binary.BigEndian.Uint64(u64)
return math.Float64frombits(u64BE), nil
}
func (f *ConsumeFuzzer) CreateSlice(targetSlice interface{}) error {
return f.GenerateStruct(targetSlice)
}

View File

@ -1,62 +0,0 @@
// Copyright 2023 The go-fuzz-headers Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package gofuzzheaders
import (
"fmt"
"reflect"
)
type Continue struct {
F *ConsumeFuzzer
}
func (f *ConsumeFuzzer) AddFuncs(fuzzFuncs []interface{}) {
for i := range fuzzFuncs {
v := reflect.ValueOf(fuzzFuncs[i])
if v.Kind() != reflect.Func {
panic("Need only funcs!")
}
t := v.Type()
if t.NumIn() != 2 || t.NumOut() != 1 {
fmt.Println(t.NumIn(), t.NumOut())
panic("Need 2 in and 1 out params. In must be the type. Out must be an error")
}
argT := t.In(0)
switch argT.Kind() {
case reflect.Ptr, reflect.Map:
default:
panic("fuzzFunc must take pointer or map type")
}
if t.In(1) != reflect.TypeOf(Continue{}) {
panic("fuzzFunc's second parameter must be type Continue")
}
f.Funcs[argT] = v
}
}
func (f *ConsumeFuzzer) GenerateWithCustom(targetStruct interface{}) error {
e := reflect.ValueOf(targetStruct).Elem()
return f.fuzzStruct(e, true)
}
func (c Continue) GenerateStruct(targetStruct interface{}) error {
return c.F.GenerateStruct(targetStruct)
}
func (c Continue) GenerateStructWithCustom(targetStruct interface{}) error {
return c.F.GenerateWithCustom(targetStruct)
}

View File

@ -1,556 +0,0 @@
// Copyright 2023 The go-fuzz-headers Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package gofuzzheaders
import (
"fmt"
"strings"
)
// returns a keyword by index
func getKeyword(f *ConsumeFuzzer) (string, error) {
index, err := f.GetInt()
if err != nil {
return keywords[0], err
}
for i, k := range keywords {
if i == index {
return k, nil
}
}
return keywords[0], fmt.Errorf("could not get a kw")
}
// Simple utility function to check if a string
// slice contains a string.
func containsString(s []string, e string) bool {
for _, a := range s {
if a == e {
return true
}
}
return false
}
// These keywords are used specifically for fuzzing Vitess
var keywords = []string{
"accessible", "action", "add", "after", "against", "algorithm",
"all", "alter", "always", "analyze", "and", "as", "asc", "asensitive",
"auto_increment", "avg_row_length", "before", "begin", "between",
"bigint", "binary", "_binary", "_utf8mb4", "_utf8", "_latin1", "bit",
"blob", "bool", "boolean", "both", "by", "call", "cancel", "cascade",
"cascaded", "case", "cast", "channel", "change", "char", "character",
"charset", "check", "checksum", "coalesce", "code", "collate", "collation",
"column", "columns", "comment", "committed", "commit", "compact", "complete",
"compressed", "compression", "condition", "connection", "constraint", "continue",
"convert", "copy", "cume_dist", "substr", "substring", "create", "cross",
"csv", "current_date", "current_time", "current_timestamp", "current_user",
"cursor", "data", "database", "databases", "day", "day_hour", "day_microsecond",
"day_minute", "day_second", "date", "datetime", "dec", "decimal", "declare",
"default", "definer", "delay_key_write", "delayed", "delete", "dense_rank",
"desc", "describe", "deterministic", "directory", "disable", "discard",
"disk", "distinct", "distinctrow", "div", "double", "do", "drop", "dumpfile",
"duplicate", "dynamic", "each", "else", "elseif", "empty", "enable",
"enclosed", "encryption", "end", "enforced", "engine", "engines", "enum",
"error", "escape", "escaped", "event", "exchange", "exclusive", "exists",
"exit", "explain", "expansion", "export", "extended", "extract", "false",
"fetch", "fields", "first", "first_value", "fixed", "float", "float4",
"float8", "flush", "for", "force", "foreign", "format", "from", "full",
"fulltext", "function", "general", "generated", "geometry", "geometrycollection",
"get", "global", "gtid_executed", "grant", "group", "grouping", "groups",
"group_concat", "having", "header", "high_priority", "hosts", "hour", "hour_microsecond",
"hour_minute", "hour_second", "if", "ignore", "import", "in", "index", "indexes",
"infile", "inout", "inner", "inplace", "insensitive", "insert", "insert_method",
"int", "int1", "int2", "int3", "int4", "int8", "integer", "interval",
"into", "io_after_gtids", "is", "isolation", "iterate", "invoker", "join",
"json", "json_table", "key", "keys", "keyspaces", "key_block_size", "kill", "lag",
"language", "last", "last_value", "last_insert_id", "lateral", "lead", "leading",
"leave", "left", "less", "level", "like", "limit", "linear", "lines",
"linestring", "load", "local", "localtime", "localtimestamp", "lock", "logs",
"long", "longblob", "longtext", "loop", "low_priority", "manifest",
"master_bind", "match", "max_rows", "maxvalue", "mediumblob", "mediumint",
"mediumtext", "memory", "merge", "microsecond", "middleint", "min_rows", "minute",
"minute_microsecond", "minute_second", "mod", "mode", "modify", "modifies",
"multilinestring", "multipoint", "multipolygon", "month", "name",
"names", "natural", "nchar", "next", "no", "none", "not", "no_write_to_binlog",
"nth_value", "ntile", "null", "numeric", "of", "off", "offset", "on",
"only", "open", "optimize", "optimizer_costs", "option", "optionally",
"or", "order", "out", "outer", "outfile", "over", "overwrite", "pack_keys",
"parser", "partition", "partitioning", "password", "percent_rank", "plugins",
"point", "polygon", "precision", "primary", "privileges", "processlist",
"procedure", "query", "quarter", "range", "rank", "read", "reads", "read_write",
"real", "rebuild", "recursive", "redundant", "references", "regexp", "relay",
"release", "remove", "rename", "reorganize", "repair", "repeat", "repeatable",
"replace", "require", "resignal", "restrict", "return", "retry", "revert",
"revoke", "right", "rlike", "rollback", "row", "row_format", "row_number",
"rows", "s3", "savepoint", "schema", "schemas", "second", "second_microsecond",
"security", "select", "sensitive", "separator", "sequence", "serializable",
"session", "set", "share", "shared", "show", "signal", "signed", "slow",
"smallint", "spatial", "specific", "sql", "sqlexception", "sqlstate",
"sqlwarning", "sql_big_result", "sql_cache", "sql_calc_found_rows",
"sql_no_cache", "sql_small_result", "ssl", "start", "starting",
"stats_auto_recalc", "stats_persistent", "stats_sample_pages", "status",
"storage", "stored", "straight_join", "stream", "system", "vstream",
"table", "tables", "tablespace", "temporary", "temptable", "terminated",
"text", "than", "then", "time", "timestamp", "timestampadd", "timestampdiff",
"tinyblob", "tinyint", "tinytext", "to", "trailing", "transaction", "tree",
"traditional", "trigger", "triggers", "true", "truncate", "uncommitted",
"undefined", "undo", "union", "unique", "unlock", "unsigned", "update",
"upgrade", "usage", "use", "user", "user_resources", "using", "utc_date",
"utc_time", "utc_timestamp", "validation", "values", "variables", "varbinary",
"varchar", "varcharacter", "varying", "vgtid_executed", "virtual", "vindex",
"vindexes", "view", "vitess", "vitess_keyspaces", "vitess_metadata",
"vitess_migration", "vitess_migrations", "vitess_replication_status",
"vitess_shards", "vitess_tablets", "vschema", "warnings", "when",
"where", "while", "window", "with", "without", "work", "write", "xor",
"year", "year_month", "zerofill",
}
// Keywords that could get an additional keyword
var needCustomString = []string{
"DISTINCTROW", "FROM", // Select keywords:
"GROUP BY", "HAVING", "WINDOW",
"FOR",
"ORDER BY", "LIMIT",
"INTO", "PARTITION", "AS", // Insert Keywords:
"ON DUPLICATE KEY UPDATE",
"WHERE", "LIMIT", // Delete keywords
"INFILE", "INTO TABLE", "CHARACTER SET", // Load keywords
"TERMINATED BY", "ENCLOSED BY",
"ESCAPED BY", "STARTING BY",
"TERMINATED BY", "STARTING BY",
"IGNORE",
"VALUE", "VALUES", // Replace tokens
"SET", // Update tokens
"ENGINE =", // Drop tokens
"DEFINER =", "ON SCHEDULE", "RENAME TO", // Alter tokens
"COMMENT", "DO", "INITIAL_SIZE = ", "OPTIONS",
}
var alterTableTokens = [][]string{
{"CUSTOM_FUZZ_STRING"},
{"CUSTOM_ALTTER_TABLE_OPTIONS"},
{"PARTITION_OPTIONS_FOR_ALTER_TABLE"},
}
var alterTokens = [][]string{
{
"DATABASE", "SCHEMA", "DEFINER = ", "EVENT", "FUNCTION", "INSTANCE",
"LOGFILE GROUP", "PROCEDURE", "SERVER",
},
{"CUSTOM_FUZZ_STRING"},
{
"ON SCHEDULE", "ON COMPLETION PRESERVE", "ON COMPLETION NOT PRESERVE",
"ADD UNDOFILE", "OPTIONS",
},
{"RENAME TO", "INITIAL_SIZE = "},
{"ENABLE", "DISABLE", "DISABLE ON SLAVE", "ENGINE"},
{"COMMENT"},
{"DO"},
}
var setTokens = [][]string{
{"CHARACTER SET", "CHARSET", "CUSTOM_FUZZ_STRING", "NAMES"},
{"CUSTOM_FUZZ_STRING", "DEFAULT", "="},
{"CUSTOM_FUZZ_STRING"},
}
var dropTokens = [][]string{
{"TEMPORARY", "UNDO"},
{
"DATABASE", "SCHEMA", "EVENT", "INDEX", "LOGFILE GROUP",
"PROCEDURE", "FUNCTION", "SERVER", "SPATIAL REFERENCE SYSTEM",
"TABLE", "TABLESPACE", "TRIGGER", "VIEW",
},
{"IF EXISTS"},
{"CUSTOM_FUZZ_STRING"},
{"ON", "ENGINE = ", "RESTRICT", "CASCADE"},
}
var renameTokens = [][]string{
{"TABLE"},
{"CUSTOM_FUZZ_STRING"},
{"TO"},
{"CUSTOM_FUZZ_STRING"},
}
var truncateTokens = [][]string{
{"TABLE"},
{"CUSTOM_FUZZ_STRING"},
}
var createTokens = [][]string{
{"OR REPLACE", "TEMPORARY", "UNDO"}, // For create spatial reference system
{
"UNIQUE", "FULLTEXT", "SPATIAL", "ALGORITHM = UNDEFINED", "ALGORITHM = MERGE",
"ALGORITHM = TEMPTABLE",
},
{
"DATABASE", "SCHEMA", "EVENT", "FUNCTION", "INDEX", "LOGFILE GROUP",
"PROCEDURE", "SERVER", "SPATIAL REFERENCE SYSTEM", "TABLE", "TABLESPACE",
"TRIGGER", "VIEW",
},
{"IF NOT EXISTS"},
{"CUSTOM_FUZZ_STRING"},
}
/*
// For future use.
var updateTokens = [][]string{
{"LOW_PRIORITY"},
{"IGNORE"},
{"SET"},
{"WHERE"},
{"ORDER BY"},
{"LIMIT"},
}
*/
var replaceTokens = [][]string{
{"LOW_PRIORITY", "DELAYED"},
{"INTO"},
{"PARTITION"},
{"CUSTOM_FUZZ_STRING"},
{"VALUES", "VALUE"},
}
var loadTokens = [][]string{
{"DATA"},
{"LOW_PRIORITY", "CONCURRENT", "LOCAL"},
{"INFILE"},
{"REPLACE", "IGNORE"},
{"INTO TABLE"},
{"PARTITION"},
{"CHARACTER SET"},
{"FIELDS", "COLUMNS"},
{"TERMINATED BY"},
{"OPTIONALLY"},
{"ENCLOSED BY"},
{"ESCAPED BY"},
{"LINES"},
{"STARTING BY"},
{"TERMINATED BY"},
{"IGNORE"},
{"LINES", "ROWS"},
{"CUSTOM_FUZZ_STRING"},
}
// These Are everything that comes after "INSERT"
var insertTokens = [][]string{
{"LOW_PRIORITY", "DELAYED", "HIGH_PRIORITY", "IGNORE"},
{"INTO"},
{"PARTITION"},
{"CUSTOM_FUZZ_STRING"},
{"AS"},
{"ON DUPLICATE KEY UPDATE"},
}
// These are everything that comes after "SELECT"
var selectTokens = [][]string{
{"*", "CUSTOM_FUZZ_STRING", "DISTINCTROW"},
{"HIGH_PRIORITY"},
{"STRAIGHT_JOIN"},
{"SQL_SMALL_RESULT", "SQL_BIG_RESULT", "SQL_BUFFER_RESULT"},
{"SQL_NO_CACHE", "SQL_CALC_FOUND_ROWS"},
{"CUSTOM_FUZZ_STRING"},
{"FROM"},
{"WHERE"},
{"GROUP BY"},
{"HAVING"},
{"WINDOW"},
{"ORDER BY"},
{"LIMIT"},
{"CUSTOM_FUZZ_STRING"},
{"FOR"},
}
// These are everything that comes after "DELETE"
var deleteTokens = [][]string{
{"LOW_PRIORITY", "QUICK", "IGNORE", "FROM", "AS"},
{"PARTITION"},
{"WHERE"},
{"ORDER BY"},
{"LIMIT"},
}
var alter_table_options = []string{
"ADD", "COLUMN", "FIRST", "AFTER", "INDEX", "KEY", "FULLTEXT", "SPATIAL",
"CONSTRAINT", "UNIQUE", "FOREIGN KEY", "CHECK", "ENFORCED", "DROP", "ALTER",
"NOT", "INPLACE", "COPY", "SET", "VISIBLE", "INVISIBLE", "DEFAULT", "CHANGE",
"CHARACTER SET", "COLLATE", "DISABLE", "ENABLE", "KEYS", "TABLESPACE", "LOCK",
"FORCE", "MODIFY", "SHARED", "EXCLUSIVE", "NONE", "ORDER BY", "RENAME COLUMN",
"AS", "=", "ASC", "DESC", "WITH", "WITHOUT", "VALIDATION", "ADD PARTITION",
"DROP PARTITION", "DISCARD PARTITION", "IMPORT PARTITION", "TRUNCATE PARTITION",
"COALESCE PARTITION", "REORGANIZE PARTITION", "EXCHANGE PARTITION",
"ANALYZE PARTITION", "CHECK PARTITION", "OPTIMIZE PARTITION", "REBUILD PARTITION",
"REPAIR PARTITION", "REMOVE PARTITIONING", "USING", "BTREE", "HASH", "COMMENT",
"KEY_BLOCK_SIZE", "WITH PARSER", "AUTOEXTEND_SIZE", "AUTO_INCREMENT", "AVG_ROW_LENGTH",
"CHECKSUM", "INSERT_METHOD", "ROW_FORMAT", "DYNAMIC", "FIXED", "COMPRESSED", "REDUNDANT",
"COMPACT", "SECONDARY_ENGINE_ATTRIBUTE", "STATS_AUTO_RECALC", "STATS_PERSISTENT",
"STATS_SAMPLE_PAGES", "ZLIB", "LZ4", "ENGINE_ATTRIBUTE", "KEY_BLOCK_SIZE", "MAX_ROWS",
"MIN_ROWS", "PACK_KEYS", "PASSWORD", "COMPRESSION", "CONNECTION", "DIRECTORY",
"DELAY_KEY_WRITE", "ENCRYPTION", "STORAGE", "DISK", "MEMORY", "UNION",
}
// Creates an 'alter table' statement. 'alter table' is an exception
// in that it has its own function. The majority of statements
// are created by 'createStmt()'.
func createAlterTableStmt(f *ConsumeFuzzer) (string, error) {
maxArgs, err := f.GetInt()
if err != nil {
return "", err
}
maxArgs = maxArgs % 30
if maxArgs == 0 {
return "", fmt.Errorf("could not create alter table stmt")
}
var stmt strings.Builder
stmt.WriteString("ALTER TABLE ")
for i := 0; i < maxArgs; i++ {
// Calculate if we get existing token or custom string
tokenType, err := f.GetInt()
if err != nil {
return "", err
}
if tokenType%4 == 1 {
customString, err := f.GetString()
if err != nil {
return "", err
}
stmt.WriteString(" " + customString)
} else {
tokenIndex, err := f.GetInt()
if err != nil {
return "", err
}
stmt.WriteString(" " + alter_table_options[tokenIndex%len(alter_table_options)])
}
}
return stmt.String(), nil
}
func chooseToken(tokens []string, f *ConsumeFuzzer) (string, error) {
index, err := f.GetInt()
if err != nil {
return "", err
}
var token strings.Builder
token.WriteString(tokens[index%len(tokens)])
if token.String() == "CUSTOM_FUZZ_STRING" {
customFuzzString, err := f.GetString()
if err != nil {
return "", err
}
return customFuzzString, nil
}
// Check if token requires an argument
if containsString(needCustomString, token.String()) {
customFuzzString, err := f.GetString()
if err != nil {
return "", err
}
token.WriteString(" " + customFuzzString)
}
return token.String(), nil
}
var stmtTypes = map[string][][]string{
"DELETE": deleteTokens,
"INSERT": insertTokens,
"SELECT": selectTokens,
"LOAD": loadTokens,
"REPLACE": replaceTokens,
"CREATE": createTokens,
"DROP": dropTokens,
"RENAME": renameTokens,
"TRUNCATE": truncateTokens,
"SET": setTokens,
"ALTER": alterTokens,
"ALTER TABLE": alterTableTokens, // ALTER TABLE has its own set of tokens
}
var stmtTypeEnum = map[int]string{
0: "DELETE",
1: "INSERT",
2: "SELECT",
3: "LOAD",
4: "REPLACE",
5: "CREATE",
6: "DROP",
7: "RENAME",
8: "TRUNCATE",
9: "SET",
10: "ALTER",
11: "ALTER TABLE",
}
func createStmt(f *ConsumeFuzzer) (string, error) {
stmtIndex, err := f.GetInt()
if err != nil {
return "", err
}
stmtIndex = stmtIndex % len(stmtTypes)
queryType := stmtTypeEnum[stmtIndex]
tokens := stmtTypes[queryType]
// We have custom creator for ALTER TABLE
if queryType == "ALTER TABLE" {
query, err := createAlterTableStmt(f)
if err != nil {
return "", err
}
return query, nil
}
// Here we are creating a query that is not
// an 'alter table' query. For available
// queries, see "stmtTypes"
// First specify the first query keyword:
var query strings.Builder
query.WriteString(queryType)
// Next create the args for the
queryArgs, err := createStmtArgs(tokens, f)
if err != nil {
return "", err
}
query.WriteString(" " + queryArgs)
return query.String(), nil
}
// Creates the arguments of a statements. In a select statement
// that would be everything after "select".
func createStmtArgs(tokenslice [][]string, f *ConsumeFuzzer) (string, error) {
var query, token strings.Builder
// We go through the tokens in the tokenslice,
// create the respective token and add it to
// "query"
for _, tokens := range tokenslice {
// For extra randomization, the fuzzer can
// choose to not include this token.
includeThisToken, err := f.GetBool()
if err != nil {
return "", err
}
if !includeThisToken {
continue
}
// There may be several tokens to choose from:
if len(tokens) > 1 {
chosenToken, err := chooseToken(tokens, f)
if err != nil {
return "", err
}
query.WriteString(" " + chosenToken)
} else {
token.WriteString(tokens[0])
// In case the token is "CUSTOM_FUZZ_STRING"
// we will then create a non-structured string
if token.String() == "CUSTOM_FUZZ_STRING" {
customFuzzString, err := f.GetString()
if err != nil {
return "", err
}
query.WriteString(" " + customFuzzString)
continue
}
// Check if token requires an argument.
// Tokens that take an argument can be found
// in 'needCustomString'. If so, we add a
// non-structured string to the token.
if containsString(needCustomString, token.String()) {
customFuzzString, err := f.GetString()
if err != nil {
return "", err
}
token.WriteString(fmt.Sprintf(" %s", customFuzzString))
}
query.WriteString(fmt.Sprintf(" %s", token.String()))
}
}
return query.String(), nil
}
// Creates a semi-structured query. It creates a string
// that is a combination of the keywords and random strings.
func createQuery(f *ConsumeFuzzer) (string, error) {
queryLen, err := f.GetInt()
if err != nil {
return "", err
}
maxLen := queryLen % 60
if maxLen == 0 {
return "", fmt.Errorf("could not create a query")
}
var query strings.Builder
for i := 0; i < maxLen; i++ {
// Get a new token:
useKeyword, err := f.GetBool()
if err != nil {
return "", err
}
if useKeyword {
keyword, err := getKeyword(f)
if err != nil {
return "", err
}
query.WriteString(" " + keyword)
} else {
customString, err := f.GetString()
if err != nil {
return "", err
}
query.WriteString(" " + customString)
}
}
if query.String() == "" {
return "", fmt.Errorf("could not create a query")
}
return query.String(), nil
}
// GetSQLString is the API that users interact with.
//
// Usage:
//
// f := NewConsumer(data)
// sqlString, err := f.GetSQLString()
func (f *ConsumeFuzzer) GetSQLString() (string, error) {
var query string
veryStructured, err := f.GetBool()
if err != nil {
return "", err
}
if veryStructured {
query, err = createStmt(f)
if err != nil {
return "", err
}
} else {
query, err = createQuery(f)
if err != nil {
return "", err
}
}
return query, nil
}

View File

@ -1,207 +0,0 @@
package testing
import (
"fmt"
fuzz "github.com/AdaLogics/go-fuzz-headers"
"os"
"reflect"
)
type F struct {
Data []byte
T *T
FuzzFunc func(*T, any)
}
func (f *F) CleanupTempDirs() {
f.T.CleanupTempDirs()
}
func (f *F) Add(args ...any) {}
func (c *F) Cleanup(f func()) {}
func (c *F) Error(args ...any) {}
func (c *F) Errorf(format string, args ...any) {}
func (f *F) Fail() {}
func (c *F) FailNow() {}
func (c *F) Failed() bool { return false }
func (c *F) Fatal(args ...any) {}
func (c *F) Fatalf(format string, args ...any) {}
func (f *F) Fuzz(ff any) {
// we are assuming that ff is a func.
// TODO: Add a check for UX purposes
fn := reflect.ValueOf(ff)
fnType := fn.Type()
var types []reflect.Type
for i := 1; i < fnType.NumIn(); i++ {
t := fnType.In(i)
types = append(types, t)
}
args := []reflect.Value{reflect.ValueOf(f.T)}
fuzzConsumer := fuzz.NewConsumer(f.Data)
for _, v := range types {
switch v.String() {
case "[]uint8":
b, err := fuzzConsumer.GetBytes()
if err != nil {
return
}
newBytes := reflect.New(v)
newBytes.Elem().SetBytes(b)
args = append(args, newBytes.Elem())
case "string":
s, err := fuzzConsumer.GetString()
if err != nil {
return
}
newString := reflect.New(v)
newString.Elem().SetString(s)
args = append(args, newString.Elem())
case "int":
randInt, err := fuzzConsumer.GetInt()
if err != nil {
return
}
newInt := reflect.New(v)
newInt.Elem().SetInt(int64(randInt))
args = append(args, newInt.Elem())
case "int8":
randInt, err := fuzzConsumer.GetInt()
if err != nil {
return
}
newInt := reflect.New(v)
newInt.Elem().SetInt(int64(randInt))
args = append(args, newInt.Elem())
case "int16":
randInt, err := fuzzConsumer.GetInt()
if err != nil {
return
}
newInt := reflect.New(v)
newInt.Elem().SetInt(int64(randInt))
args = append(args, newInt.Elem())
case "int32":
randInt, err := fuzzConsumer.GetInt()
if err != nil {
return
}
newInt := reflect.New(v)
newInt.Elem().SetInt(int64(randInt))
args = append(args, newInt.Elem())
case "int64":
randInt, err := fuzzConsumer.GetInt()
if err != nil {
return
}
newInt := reflect.New(v)
newInt.Elem().SetInt(int64(randInt))
args = append(args, newInt.Elem())
case "uint":
randInt, err := fuzzConsumer.GetInt()
if err != nil {
return
}
newUint := reflect.New(v)
newUint.Elem().SetUint(uint64(randInt))
args = append(args, newUint.Elem())
case "uint8":
randInt, err := fuzzConsumer.GetInt()
if err != nil {
return
}
newUint := reflect.New(v)
newUint.Elem().SetUint(uint64(randInt))
args = append(args, newUint.Elem())
case "uint16":
randInt, err := fuzzConsumer.GetUint16()
if err != nil {
return
}
newUint16 := reflect.New(v)
newUint16.Elem().SetUint(uint64(randInt))
args = append(args, newUint16.Elem())
case "uint32":
randInt, err := fuzzConsumer.GetUint32()
if err != nil {
return
}
newUint32 := reflect.New(v)
newUint32.Elem().SetUint(uint64(randInt))
args = append(args, newUint32.Elem())
case "uint64":
randInt, err := fuzzConsumer.GetUint64()
if err != nil {
return
}
newUint64 := reflect.New(v)
newUint64.Elem().SetUint(uint64(randInt))
args = append(args, newUint64.Elem())
case "rune":
randRune, err := fuzzConsumer.GetRune()
if err != nil {
return
}
newRune := reflect.New(v)
newRune.Elem().Set(reflect.ValueOf(randRune))
args = append(args, newRune.Elem())
case "float32":
randFloat, err := fuzzConsumer.GetFloat32()
if err != nil {
return
}
newFloat := reflect.New(v)
newFloat.Elem().Set(reflect.ValueOf(randFloat))
args = append(args, newFloat.Elem())
case "float64":
randFloat, err := fuzzConsumer.GetFloat64()
if err != nil {
return
}
newFloat := reflect.New(v)
newFloat.Elem().Set(reflect.ValueOf(randFloat))
args = append(args, newFloat.Elem())
case "bool":
randBool, err := fuzzConsumer.GetBool()
if err != nil {
return
}
newBool := reflect.New(v)
newBool.Elem().Set(reflect.ValueOf(randBool))
args = append(args, newBool.Elem())
default:
fmt.Println(v.String())
}
}
fn.Call(args)
}
func (f *F) Helper() {}
func (c *F) Log(args ...any) {
fmt.Println(args...)
}
func (c *F) Logf(format string, args ...any) {
fmt.Println(format, args)
}
func (c *F) Name() string { return "libFuzzer" }
func (c *F) Setenv(key, value string) {}
func (c *F) Skip(args ...any) {
panic("GO-FUZZ-BUILD-PANIC")
}
func (c *F) SkipNow() {
panic("GO-FUZZ-BUILD-PANIC")
}
func (c *F) Skipf(format string, args ...any) {
panic("GO-FUZZ-BUILD-PANIC")
}
func (f *F) Skipped() bool { return false }
func (f *F) TempDir() string {
dir, err := os.MkdirTemp("", "fuzzdir-")
if err != nil {
panic(err)
}
f.T.TempDirs = append(f.T.TempDirs, dir)
return dir
}

View File

@ -1,129 +0,0 @@
package testing
import (
"fmt"
"os"
"strings"
"time"
)
// T can be used to terminate the current fuzz iteration
// without terminating the whole fuzz run. To do so, simply
// panic with the text "GO-FUZZ-BUILD-PANIC" and the fuzzer
// will recover.
type T struct {
TempDirs []string
}
func NewT() *T {
tempDirs := make([]string, 0)
return &T{TempDirs: tempDirs}
}
func unsupportedApi(name string) string {
plsOpenIss := "Please open an issue https://github.com/AdamKorcz/go-118-fuzz-build if you need this feature."
var b strings.Builder
b.WriteString(fmt.Sprintf("%s is not supported when fuzzing in libFuzzer mode\n.", name))
b.WriteString(plsOpenIss)
return b.String()
}
func (t *T) Cleanup(f func()) {
f()
}
func (t *T) Deadline() (deadline time.Time, ok bool) {
panic(unsupportedApi("t.Deadline()"))
}
func (t *T) Error(args ...any) {
fmt.Println(args...)
panic("error")
}
func (t *T) Errorf(format string, args ...any) {
fmt.Printf(format+"\n", args...)
panic("errorf")
}
func (t *T) Fail() {
panic("Called T.Fail()")
}
func (t *T) FailNow() {
panic("Called T.Fail()")
panic(unsupportedApi("t.FailNow()"))
}
func (t *T) Failed() bool {
panic(unsupportedApi("t.Failed()"))
}
func (t *T) Fatal(args ...any) {
fmt.Println(args...)
panic("fatal")
}
func (t *T) Fatalf(format string, args ...any) {
fmt.Printf(format+"\n", args...)
panic("fatal")
}
func (t *T) Helper() {
// We can't support it, but it also just impacts how failures are reported, so we can ignore it
}
func (t *T) Log(args ...any) {
fmt.Println(args...)
}
func (t *T) Logf(format string, args ...any) {
fmt.Println(format)
fmt.Println(args...)
}
func (t *T) Name() string {
return "libFuzzer"
}
func (t *T) Parallel() {
panic(unsupportedApi("t.Parallel()"))
}
func (t *T) Run(name string, f func(t *T)) bool {
panic(unsupportedApi("t.Run()"))
}
func (t *T) Setenv(key, value string) {
}
func (t *T) Skip(args ...any) {
panic("GO-FUZZ-BUILD-PANIC")
}
func (t *T) SkipNow() {
panic("GO-FUZZ-BUILD-PANIC")
}
// Is not really supported. We just skip instead
// of printing any message. A log message can be
// added if need be.
func (t *T) Skipf(format string, args ...any) {
panic("GO-FUZZ-BUILD-PANIC")
}
func (t *T) Skipped() bool {
panic(unsupportedApi("t.Skipped()"))
}
func (t *T) TempDir() string {
dir, err := os.MkdirTemp("", "fuzzdir-")
if err != nil {
panic(err)
}
t.TempDirs = append(t.TempDirs, dir)
return dir
}
func (t *T) CleanupTempDirs() {
if len(t.TempDirs) > 0 {
for _, tempDir := range t.TempDirs {
os.RemoveAll(tempDir)
}
}
}

View File

@ -1,42 +0,0 @@
package testing
import (
"testing"
)
func AllocsPerRun(runs int, f func()) (avg float64) {
panic(unsupportedApi("testing.AllocsPerRun"))
}
func CoverMode() string {
panic(unsupportedApi("testing.CoverMode"))
}
func Coverage() float64 {
panic(unsupportedApi("testing.Coverage"))
}
func Init() {
panic(unsupportedApi("testing.Init"))
}
func RegisterCover(c testing.Cover) {
panic(unsupportedApi("testing.RegisterCover"))
}
func RunExamples(matchString func(pat, str string) (bool, error), examples []testing.InternalExample) (ok bool) {
panic(unsupportedApi("testing.RunExamples"))
}
func RunTests(matchString func(pat, str string) (bool, error), tests []testing.InternalTest) (ok bool) {
panic(unsupportedApi("testing.RunTests"))
}
func Short() bool {
return false
}
func Verbose() bool {
panic(unsupportedApi("testing.Verbose"))
}
type M struct {}
func (m *M) Run() (code int) {
panic("testing.M is not support in libFuzzer Mode")
}

View File

@ -1 +0,0 @@
* text=auto eol=lf

View File

@ -1,10 +1 @@
.vscode/
*.exe
# testing
testdata
# go workspaces
go.work
go.work.sum

View File

@ -1,147 +0,0 @@
linters:
enable:
# style
- containedctx # struct contains a context
- dupl # duplicate code
- errname # erorrs are named correctly
- nolintlint # "//nolint" directives are properly explained
- revive # golint replacement
- unconvert # unnecessary conversions
- wastedassign
# bugs, performance, unused, etc ...
- contextcheck # function uses a non-inherited context
- errorlint # errors not wrapped for 1.13
- exhaustive # check exhaustiveness of enum switch statements
- gofmt # files are gofmt'ed
- gosec # security
- nilerr # returns nil even with non-nil error
- thelper # test helpers without t.Helper()
- unparam # unused function params
issues:
exclude-dirs:
- pkg/etw/sample
exclude-rules:
# err is very often shadowed in nested scopes
- linters:
- govet
text: '^shadow: declaration of "err" shadows declaration'
# ignore long lines for skip autogen directives
- linters:
- revive
text: "^line-length-limit: "
source: "^//(go:generate|sys) "
#TODO: remove after upgrading to go1.18
# ignore comment spacing for nolint and sys directives
- linters:
- revive
text: "^comment-spacings: no space between comment delimiter and comment text"
source: "//(cspell:|nolint:|sys |todo)"
# not on go 1.18 yet, so no any
- linters:
- revive
text: "^use-any: since GO 1.18 'interface{}' can be replaced by 'any'"
# allow unjustified ignores of error checks in defer statements
- linters:
- nolintlint
text: "^directive `//nolint:errcheck` should provide explanation"
source: '^\s*defer '
# allow unjustified ignores of error lints for io.EOF
- linters:
- nolintlint
text: "^directive `//nolint:errorlint` should provide explanation"
source: '[=|!]= io.EOF'
linters-settings:
exhaustive:
default-signifies-exhaustive: true
govet:
enable-all: true
disable:
# struct order is often for Win32 compat
# also, ignore pointer bytes/GC issues for now until performance becomes an issue
- fieldalignment
nolintlint:
require-explanation: true
require-specific: true
revive:
# revive is more configurable than static check, so likely the preferred alternative to static-check
# (once the perf issue is solved: https://github.com/golangci/golangci-lint/issues/2997)
enable-all-rules:
true
# https://github.com/mgechev/revive/blob/master/RULES_DESCRIPTIONS.md
rules:
# rules with required arguments
- name: argument-limit
disabled: true
- name: banned-characters
disabled: true
- name: cognitive-complexity
disabled: true
- name: cyclomatic
disabled: true
- name: file-header
disabled: true
- name: function-length
disabled: true
- name: function-result-limit
disabled: true
- name: max-public-structs
disabled: true
# geneally annoying rules
- name: add-constant # complains about any and all strings and integers
disabled: true
- name: confusing-naming # we frequently use "Foo()" and "foo()" together
disabled: true
- name: flag-parameter # excessive, and a common idiom we use
disabled: true
- name: unhandled-error # warns over common fmt.Print* and io.Close; rely on errcheck instead
disabled: true
# general config
- name: line-length-limit
arguments:
- 140
- name: var-naming
arguments:
- []
- - CID
- CRI
- CTRD
- DACL
- DLL
- DOS
- ETW
- FSCTL
- GCS
- GMSA
- HCS
- HV
- IO
- LCOW
- LDAP
- LPAC
- LTSC
- MMIO
- NT
- OCI
- PMEM
- PWSH
- RX
- SACl
- SID
- SMB
- TX
- VHD
- VHDX
- VMID
- VPCI
- WCOW
- WIM

View File

@ -1 +0,0 @@
* @microsoft/containerplat

View File

@ -1,4 +1,4 @@
# go-winio [![Build Status](https://github.com/microsoft/go-winio/actions/workflows/ci.yml/badge.svg)](https://github.com/microsoft/go-winio/actions/workflows/ci.yml)
# go-winio
This repository contains utilities for efficiently performing Win32 IO operations in
Go. Currently, this is focused on accessing named pipes and other file handles, and
@ -11,79 +11,12 @@ package.
Please see the LICENSE file for licensing information.
## Contributing
This project has adopted the [Microsoft Open Source Code of
Conduct](https://opensource.microsoft.com/codeofconduct/). For more information
see the [Code of Conduct
FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact
[opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional
questions or comments.
This project welcomes contributions and suggestions.
Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that
you have the right to, and actually do, grant us the rights to use your contribution.
For details, visit [Microsoft CLA](https://cla.microsoft.com).
When you submit a pull request, a CLA-bot will automatically determine whether you need to
provide a CLA and decorate the PR appropriately (e.g., label, comment).
Simply follow the instructions provided by the bot.
You will only need to do this once across all repos using our CLA.
Additionally, the pull request pipeline requires the following steps to be performed before
mergining.
### Code Sign-Off
We require that contributors sign their commits using [`git commit --signoff`][git-commit-s]
to certify they either authored the work themselves or otherwise have permission to use it in this project.
A range of commits can be signed off using [`git rebase --signoff`][git-rebase-s].
Please see [the developer certificate](https://developercertificate.org) for more info,
as well as to make sure that you can attest to the rules listed.
Our CI uses the DCO Github app to ensure that all commits in a given PR are signed-off.
### Linting
Code must pass a linting stage, which uses [`golangci-lint`][lint].
The linting settings are stored in [`.golangci.yaml`](./.golangci.yaml), and can be run
automatically with VSCode by adding the following to your workspace or folder settings:
```json
"go.lintTool": "golangci-lint",
"go.lintOnSave": "package",
```
Additional editor [integrations options are also available][lint-ide].
Alternatively, `golangci-lint` can be [installed locally][lint-install] and run from the repo root:
```shell
# use . or specify a path to only lint a package
# to show all lint errors, use flags "--max-issues-per-linter=0 --max-same-issues=0"
> golangci-lint run ./...
```
### Go Generate
The pipeline checks that auto-generated code, via `go generate`, are up to date.
This can be done for the entire repo:
```shell
> go generate ./...
```
## Code of Conduct
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
## Special Thanks
Thanks to [natefinch][natefinch] for the inspiration for this library.
See [npipe](https://github.com/natefinch/npipe) for another named pipe implementation.
[lint]: https://golangci-lint.run/
[lint-ide]: https://golangci-lint.run/usage/integrations/#editor-integration
[lint-install]: https://golangci-lint.run/usage/install/#local-installation
[git-commit-s]: https://git-scm.com/docs/git-commit#Documentation/git-commit.txt--s
[git-rebase-s]: https://git-scm.com/docs/git-rebase#Documentation/git-rebase.txt---signoff
[natefinch]: https://github.com/natefinch
Thanks to natefinch for the inspiration for this library. See https://github.com/natefinch/npipe
for another named pipe implementation.

View File

@ -1,41 +0,0 @@
<!-- BEGIN MICROSOFT SECURITY.MD V0.0.7 BLOCK -->
## Security
Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/).
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/opensource/security/definition), please report it to us as described below.
## Reporting Security Issues
**Please do not report security vulnerabilities through public GitHub issues.**
Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/opensource/security/create-report).
If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/opensource/security/pgpkey).
You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://aka.ms/opensource/security/msrc).
Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:
* Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.)
* Full paths of source file(s) related to the manifestation of the issue
* The location of the affected source code (tag/branch/commit or direct URL)
* Any special configuration required to reproduce the issue
* Step-by-step instructions to reproduce the issue
* Proof-of-concept or exploit code (if possible)
* Impact of the issue, including how an attacker might exploit the issue
This information will help us triage your report more quickly.
If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/opensource/security/bounty) page for more details about our active programs.
## Preferred Languages
We prefer all communications to be in English.
## Policy
Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/opensource/security/cvd).
<!-- END MICROSOFT SECURITY.MD BLOCK -->

View File

@ -1,4 +1,4 @@
Copyright (c) 2017 The Go Authors. All rights reserved.
Copyright (c) 2012 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are

View File

@ -1,4 +1,3 @@
//go:build windows
// +build windows
package winio
@ -8,16 +7,15 @@ import (
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"runtime"
"syscall"
"unicode/utf16"
"github.com/Microsoft/go-winio/internal/fs"
"golang.org/x/sys/windows"
)
//sys backupRead(h windows.Handle, b []byte, bytesRead *uint32, abort bool, processSecurity bool, context *uintptr) (err error) = BackupRead
//sys backupWrite(h windows.Handle, b []byte, bytesWritten *uint32, abort bool, processSecurity bool, context *uintptr) (err error) = BackupWrite
//sys backupRead(h syscall.Handle, b []byte, bytesRead *uint32, abort bool, processSecurity bool, context *uintptr) (err error) = BackupRead
//sys backupWrite(h syscall.Handle, b []byte, bytesWritten *uint32, abort bool, processSecurity bool, context *uintptr) (err error) = BackupWrite
const (
BackupData = uint32(iota + 1)
@ -26,7 +24,7 @@ const (
BackupAlternateData
BackupLink
BackupPropertyData
BackupObjectId //revive:disable-line:var-naming ID, not Id
BackupObjectId
BackupReparseData
BackupSparseBlock
BackupTxfsData
@ -36,16 +34,14 @@ const (
StreamSparseAttributes = uint32(8)
)
//nolint:revive // var-naming: ALL_CAPS
const (
WRITE_DAC = windows.WRITE_DAC
WRITE_OWNER = windows.WRITE_OWNER
ACCESS_SYSTEM_SECURITY = windows.ACCESS_SYSTEM_SECURITY
WRITE_DAC = 0x40000
WRITE_OWNER = 0x80000
ACCESS_SYSTEM_SECURITY = 0x1000000
)
// BackupHeader represents a backup stream of a file.
type BackupHeader struct {
//revive:disable-next-line:var-naming ID, not Id
Id uint32 // The backup stream ID
Attributes uint32 // Stream attributes
Size int64 // The size of the stream in bytes
@ -53,8 +49,8 @@ type BackupHeader struct {
Offset int64 // The offset of the stream in the file (for BackupSparseBlock only).
}
type win32StreamID struct {
StreamID uint32
type win32StreamId struct {
StreamId uint32
Attributes uint32
Size uint64
NameSize uint32
@ -75,7 +71,7 @@ func NewBackupStreamReader(r io.Reader) *BackupStreamReader {
// Next returns the next backup stream and prepares for calls to Read(). It skips the remainder of the current stream if
// it was not completely read.
func (r *BackupStreamReader) Next() (*BackupHeader, error) {
if r.bytesLeft > 0 { //nolint:nestif // todo: flatten this
if r.bytesLeft > 0 {
if s, ok := r.r.(io.Seeker); ok {
// Make sure Seek on io.SeekCurrent sometimes succeeds
// before trying the actual seek.
@ -86,16 +82,16 @@ func (r *BackupStreamReader) Next() (*BackupHeader, error) {
r.bytesLeft = 0
}
}
if _, err := io.Copy(io.Discard, r); err != nil {
if _, err := io.Copy(ioutil.Discard, r); err != nil {
return nil, err
}
}
var wsi win32StreamID
var wsi win32StreamId
if err := binary.Read(r.r, binary.LittleEndian, &wsi); err != nil {
return nil, err
}
hdr := &BackupHeader{
Id: wsi.StreamID,
Id: wsi.StreamId,
Attributes: wsi.Attributes,
Size: int64(wsi.Size),
}
@ -104,9 +100,9 @@ func (r *BackupStreamReader) Next() (*BackupHeader, error) {
if err := binary.Read(r.r, binary.LittleEndian, name); err != nil {
return nil, err
}
hdr.Name = windows.UTF16ToString(name)
hdr.Name = syscall.UTF16ToString(name)
}
if wsi.StreamID == BackupSparseBlock {
if wsi.StreamId == BackupSparseBlock {
if err := binary.Read(r.r, binary.LittleEndian, &hdr.Offset); err != nil {
return nil, err
}
@ -151,8 +147,8 @@ func (w *BackupStreamWriter) WriteHeader(hdr *BackupHeader) error {
return fmt.Errorf("missing %d bytes", w.bytesLeft)
}
name := utf16.Encode([]rune(hdr.Name))
wsi := win32StreamID{
StreamID: hdr.Id,
wsi := win32StreamId{
StreamId: hdr.Id,
Attributes: hdr.Attributes,
Size: uint64(hdr.Size),
NameSize: uint32(len(name) * 2),
@ -205,9 +201,9 @@ func NewBackupFileReader(f *os.File, includeSecurity bool) *BackupFileReader {
// Read reads a backup stream from the file by calling the Win32 API BackupRead().
func (r *BackupFileReader) Read(b []byte) (int, error) {
var bytesRead uint32
err := backupRead(windows.Handle(r.f.Fd()), b, &bytesRead, false, r.includeSecurity, &r.ctx)
err := backupRead(syscall.Handle(r.f.Fd()), b, &bytesRead, false, r.includeSecurity, &r.ctx)
if err != nil {
return 0, &os.PathError{Op: "BackupRead", Path: r.f.Name(), Err: err}
return 0, &os.PathError{"BackupRead", r.f.Name(), err}
}
runtime.KeepAlive(r.f)
if bytesRead == 0 {
@ -220,7 +216,7 @@ func (r *BackupFileReader) Read(b []byte) (int, error) {
// the underlying file.
func (r *BackupFileReader) Close() error {
if r.ctx != 0 {
_ = backupRead(windows.Handle(r.f.Fd()), nil, nil, true, false, &r.ctx)
backupRead(syscall.Handle(r.f.Fd()), nil, nil, true, false, &r.ctx)
runtime.KeepAlive(r.f)
r.ctx = 0
}
@ -244,9 +240,9 @@ func NewBackupFileWriter(f *os.File, includeSecurity bool) *BackupFileWriter {
// Write restores a portion of the file using the provided backup stream.
func (w *BackupFileWriter) Write(b []byte) (int, error) {
var bytesWritten uint32
err := backupWrite(windows.Handle(w.f.Fd()), b, &bytesWritten, false, w.includeSecurity, &w.ctx)
err := backupWrite(syscall.Handle(w.f.Fd()), b, &bytesWritten, false, w.includeSecurity, &w.ctx)
if err != nil {
return 0, &os.PathError{Op: "BackupWrite", Path: w.f.Name(), Err: err}
return 0, &os.PathError{"BackupWrite", w.f.Name(), err}
}
runtime.KeepAlive(w.f)
if int(bytesWritten) != len(b) {
@ -259,7 +255,7 @@ func (w *BackupFileWriter) Write(b []byte) (int, error) {
// close the underlying file.
func (w *BackupFileWriter) Close() error {
if w.ctx != 0 {
_ = backupWrite(windows.Handle(w.f.Fd()), nil, nil, true, false, &w.ctx)
backupWrite(syscall.Handle(w.f.Fd()), nil, nil, true, false, &w.ctx)
runtime.KeepAlive(w.f)
w.ctx = 0
}
@ -271,14 +267,11 @@ func (w *BackupFileWriter) Close() error {
//
// If the file opened was a directory, it cannot be used with Readdir().
func OpenForBackup(path string, access uint32, share uint32, createmode uint32) (*os.File, error) {
h, err := fs.CreateFile(path,
fs.AccessMask(access),
fs.FileShareMode(share),
nil,
fs.FileCreationDisposition(createmode),
fs.FILE_FLAG_BACKUP_SEMANTICS|fs.FILE_FLAG_OPEN_REPARSE_POINT,
0,
)
winPath, err := syscall.UTF16FromString(path)
if err != nil {
return nil, err
}
h, err := syscall.CreateFile(&winPath[0], access, share, nil, createmode, syscall.FILE_FLAG_BACKUP_SEMANTICS|syscall.FILE_FLAG_OPEN_REPARSE_POINT, 0)
if err != nil {
err = &os.PathError{Op: "open", Path: path, Err: err}
return nil, err

View File

@ -1,3 +0,0 @@
// This file only exists to allow go get on non-Windows platforms.
package backuptar

Some files were not shown because too many files have changed in this diff Show More