This is an optimization that uses the results cache created by
CNI on the filesystem to store and fetch IP addresses for
containers in the core services and for functions. As part of
the change, the dependency on the syscall code from Weave net
has been removed, and the code should compile on MacOS again.
Updates and rebases the work in #38 by carlosedp
Tested in the original PR, further testing in the incoming
PR.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
This commit adds fprocess to the return values from faasd provider
/system/functions and /system/function/{name}
This has unit tests and has been tested on a faasd installation
Signed-off-by: Alistair Hey <alistair@heyal.co.uk>
This commit adds the EnvVars set on the process to the retuurn from the
faasd provider. It gets the container process and then filters out PATH
and fprocess (if found) and returns the remaining envVars as a map.
This has using tests for getting the EnvVars from procees.env and has
been tested on amd_64 faasd by building, deploying and using curl
against the provider and gateway.
Signed-off-by: Alistair Hey <alistair@heyal.co.uk>
This commit allows the provider to return a list of the names of the
secrets mapped into an openfaas function. This was tested by building
and deploying faasd on multipass and curling the provider directly and
seeing the returned secrets list!
Signed-off-by: Alistair Hey <alistair@heyal.co.uk>
All services like docker and k8s.io use their own namespaces
for core services, this change moves openfaas services into
the openfaas namespace instead of the default one.
The main change is that logs will look like:
journalctl -t openfaas:gateway
Instead of "default:gateway"
Function logs will remain unaffected and scheduled in the
openfaas-fn namespace.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
Custom containers in the compose file can have a directory
mounted to store state for things like a database. This requires
a specific user since influxdb/postgresql and other containers
create folders and update permissions on start-up.
Tested with influxdb on Ubuntu with userid 1000, which failed
before the change.
Adds a grace period in the e2e tests.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
The update flow used to delete the active function before
synchronously pulling the next and starting it. That meant
functions would always face downtime during the pull.
This changes the order to pre-pull and reduce any down time.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
By moving to a reference instead of a value type, this patch
means the annotations are not leaked into other objects in
the result of ListFunctions. Tested on x86 with a Linux host
and I could no longer reproduce the issue in #128
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
Provide support for annotations in faasd with namespaced container
labels. Unit tested and confirmed with end to end test via faasd
deployed to multipass VM
Signed-off-by: Alex Tomic <atomic777@gmail.com>
Memory limits now work and a function will be killed with OOM
however, it will remain in a stopped state and will not
restart automatically.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
This patch reports stopped tasks as having zero scale, which
means the gateway will send a "scale up" request, the same
way as it does for paused containers, or those which have
no task due to a reboot of the machine.
The scale up logic will now delete the stopped task and
recreate the task.
Tested with nodeinfo and figlet on a Dell XPS with
Ubuntu 16.04. The scaling logic has been re-written, but
re-tested by manually pausing and manually removing
the task of a container.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
Logs can now be viewed with the following, adding -f to follow
the logs.
journalctl -t default:gateway
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
System containers can now be proxied to the localhost or to
all adapters using docker-compose.
Tested with NATS and Prometheus to 127.0.0.1 in multipass
and with the gateway to 0.0.0.0.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
Fixes an issue introduced in #45 which was undetected. When
users do not pass in "labels" to the deployment - or a valid
empty object, then a nil dereference causes a panic.
Fixes: #101
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
There appeared to be an issue with logs appearing #98 and #68
@LucasRoesler spent a considerable amount of time looking into
this and concluded that the faas-provider and approach we are
taking to stream logs from journalctl as a process was
working as expected.
The issue appears to have been with the proxy code and its
use of a HTTP connection. Somewhere within the code, a buffer
was holding onto the data before flushing it 20-30 seconds later
This appeared to users as if the logs were not working at all.
Before fixing, the gateway container was tested by exposing
it over an SSH tunnel and inlets tunnel, both worked as
expected. The updates have been tested on multipass with
Ubuntu 18.04 and a binary built locally.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
Graph logic moves into depgraph package and makes internal
fields inaccessible. Completes feedback from @LucasRoesler
from previous PR where the dependency graph was added for 0.9.1
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
* Adds depends_on fields to compose YAML
* Updates parsing code to copy across depends_on field to
openfaas service from compose service definition
* Adds algorithm and unit tests for finding order
* Applies order to up.go command
* Makes unit testing on MacOS possible through build directives
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
* revendor k3sup to prevent arch / OS from being printed in
the logs
* print version on startup
* bump minor CNI and containerd version for e2e tests
* revendor faas-provider for latest log printing update
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
**What**
- Use the compose-go library to read the service definitions from an
external compose file instead of building them in Go
- Add default compose file and copy during `faasd install`
- Add test for load and parse of compose file
- Make testing easier by sorting the env keys
- Allow append to instantiate the slices so that we can more easily test
for proper parsing (e.g. nil is still nil etc)
- Add the arch suffix to the compose file and set this as part of the
env when we parse the compose file. This allows faasd to dynamically
set the arch suffix used for the basic auth and the gateway images.
Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
This patch fixes a vulnerability in the secrets API, however
it is important to stress that the user must be authenticated
as the admin user on the REST API before they can attempt this.
Reported by Appsecco via email. @lucasroesler, Appsecco and
myself believe this to be of low severity.
The fix prevents directory traversal characters from being
used in secret names. If a secret name such as:
../../root/.ssh/authorized_keys were to be used, an attacker
could remove the value and write their own.
Tested with unit tests and tests are now made to run
via the CI and a new Makefile target.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
**What**
- Instead of sleeping to ensure the msg channel is populated, use an
unbuffered channel. This seems to work just as well in all the manual
tests cases
Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
**What**
- Remove the `output-fields` flag because not all journalctl versions
support it
- Add a short sleep to the start of the log stream to avoid some kind of
race/buffering condition with the Handler
Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
**What**
- journald log provider using exec to journalctl
```
journalctl -t <namespace>:<name> --output=json --since=<timestamp> <--follow> --output-fields=SYSLOG_IDENTIFIER,MESSAGE,_PID,_SOURCE_REALTIME_TIMESTAMP
```
- This can be tested manually using `faas-cli logs` as normal, e.g.
`faas-cli logs nodeinfo` should tail the last 5 mins of logs.
- Very basic tests ensuring that the `journalctl` comamand is correctly
construction and that the json log entrys are parsed correctly.
- Add simple e2e test to grep the function logs
Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
When ListFunctions populate it's function map, it should not add
functions that GetFunction returned error.
Signed-off-by: Carlos de Paula <me@carlosedp.com>
The behaviour prior to this patch caused some confusion for
users since they expected a behaviour like Swarm / Kubernetes
which always pulls images by default, even if cached. I've tested
the change and it is working as expected. By default images are
always pulled upon deployment.
To revert to the prior behaviour, simply add to faasd up:
--pull-policy=IfNotPresent
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
Adding label when a /system/function/<name> endpoint
is invoked as it was missed in the previous commit
Signed-off-by: Martin Dekov <mvdekov@gmail.com>
The collect command redirects function logs to the journal for
viewing on journalctl. faas-cli logs is not implemented as of
yet. View logs with journalctl -t openfaas-fn:FN_NAME_HERE.
Tested on Dell XPS with Ubuntu Linux. The approach takes
inspiration from the Stellar project.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
Not sure how this got reverted / affected, but was wrong. The
name "faas-containerd" is gone and not in use.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
The use of containerd and CNI functions has been refactored to reuse
the same codebase.
Added all network functionality to own directory and package. Removed
netlink and weave library in favor of using CNI plugin result files.
Rename containers handler to functions to clear-up functionality.
Signed-off-by: Carlos de Paula <me@carlosedp.com>
This appeared to prevent the provider's secret code from
creating files in its working directory. The patch makes all
code use the same permission.
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
Adds secrets support and binding of secrets at runtime to
functions. Files are written in plain-text to a 0644 permission
folder which can only be read by root and the containers
requesting the secret through the OpenFaaS API.
Tested by deploying an alpine function using "cat" as its
fprocess.
Happy to revisit at a later date and look into encryption at
rest. This should be on-par with using Kubernetes in its
default unencrypted state.
Fixes: #29
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
