Commit Graph

104 Commits

Author SHA1 Message Date
3039773fbd Rename mac file to darwin suffix
The mac reference was incorrect, it should be darwin.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-06-17 14:48:00 +01:00
5b92e7793d Move graph logic into package
Graph logic moves into depgraph package and makes internal
fields inaccessible. Completes feedback from @LucasRoesler
from previous PR where the dependency graph was added for 0.9.1

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-06-17 14:33:58 +01:00
88f1aa0433 Update docs for Graph and Node
Updates godoc and adds Add() method instead of using
append on the private slice.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-06-17 13:40:09 +01:00
2b9efd29a0 Add depends_on field for core service ordering
* Adds depends_on fields to compose YAML
* Updates parsing code to copy across depends_on field to
openfaas service from compose service definition
* Adds algorithm and unit tests for finding order
* Applies order to up.go command
* Makes unit testing on MacOS possible through build directives

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-06-17 13:40:09 +01:00
26debca616 Print version and reduce verbosity
* revendor k3sup to prevent arch / OS from being printed in
the logs
* print version on startup
* bump minor CNI and containerd version for e2e tests
* revendor faas-provider for latest log printing update

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-06-08 09:43:51 +01:00
50de0f34bb Load core faasd service definitions from compose
**What**
- Use the compose-go library to read the service definitions from an
  external compose file instead of building them in Go
- Add default compose file and copy during `faasd install`
- Add test for load and parse of compose file
- Make testing easier  by sorting the env keys
- Allow append to instantiate the slices so that we can more easily test
  for proper parsing (e.g. nil is still nil etc)
- Add the arch suffix to the compose file and set this as part of the
  env when we parse the compose file. This allows faasd to dynamically
  set the arch suffix used for the basic auth and the gateway images.

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2020-06-07 09:32:42 +01:00
35e017b526 Remove output fields from the logs test case
**What**
- Fix the test to match the new logs command builder, without the output
  fields flag

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2020-04-29 14:48:57 +01:00
e54da61283 Use quote for test failure output
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-04-29 12:23:01 +01:00
84353d0cae Format buildCmd test output
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-04-29 12:23:01 +01:00
e33a60862d Fix vulnerability in authenticated secrets API
This patch fixes a vulnerability in the secrets API, however
it is important to stress that the user must be authenticated
as the admin user on the REST API before they can attempt this.

Reported by Appsecco via email. @lucasroesler, Appsecco and
myself believe this to be of low severity.

The fix prevents directory traversal characters from being
used in secret names. If a secret name such as:
../../root/.ssh/authorized_keys were to be used, an attacker
could remove the value and write their own.

Tested with unit tests and tests are now made to run
via the CI and a new Makefile target.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-04-29 12:23:01 +01:00
19abc9f7b9 Use an unbuffered log msg channel
**What**
- Instead of sleeping to ensure the msg channel is populated, use an
  unbuffered channel. This seems to work just as well in all the manual
  tests cases

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2020-03-08 15:00:42 +00:00
cece6cf1ef Improve journalctl version compat
**What**
- Remove the `output-fields` flag because not all journalctl versions
  support it
- Add a short sleep to the start of the log stream to avoid some kind of
  race/buffering condition with the Handler

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2020-03-07 10:11:09 +00:00
22882e2643 Initial journald log provider attempt
**What**
- journald log provider using exec to journalctl
```
journalctl -t <namespace>:<name>  --output=json --since=<timestamp> <--follow> --output-fields=SYSLOG_IDENTIFIER,MESSAGE,_PID,_SOURCE_REALTIME_TIMESTAMP
```
- This can be tested manually using `faas-cli logs` as normal, e.g.
  `faas-cli logs nodeinfo` should tail the last 5 mins of logs.
- Very basic tests ensuring that the `journalctl` comamand is correctly
  construction and that the json log entrys are parsed correctly.
- Add simple e2e test to grep the function logs

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2020-03-07 10:11:09 +00:00
667d74aaf7 Skip adding function if GetFunction returns error
When ListFunctions populate it's function map, it should not add
functions that GetFunction returned error.

Signed-off-by: Carlos de Paula <me@carlosedp.com>
2020-03-07 07:25:19 +00:00
70e7e0d25a Apply gofmt
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-03-01 20:13:18 +00:00
be8574ecd0 Always pull images by default
The behaviour prior to this patch caused some confusion for
users since they expected a behaviour like Swarm / Kubernetes
which always pulls images by default, even if cached. I've tested
the change and it is working as expected. By default images are
always pulled upon deployment.

To revert to the prior behaviour, simply add to faasd up:
--pull-policy=IfNotPresent

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-03-01 20:13:18 +00:00
87c71b090f Add label for single function query
Adding label when a /system/function/<name> endpoint
is invoked as it was missed in the previous commit

Signed-off-by: Martin Dekov <mvdekov@gmail.com>
2020-02-25 07:11:44 +00:00
560c295eb0 Enable labeling containers
Enabling the faasd-provider to label containers

Signed-off-by: Martin Dekov <mvdekov@gmail.com>
2020-02-23 20:06:30 +00:00
2307fc71c5 Add log shim and collect command
The collect command redirects function logs to the journal for
viewing on journalctl. faas-cli logs is not implemented as of
yet. View logs with journalctl -t openfaas-fn:FN_NAME_HERE.

Tested on Dell XPS with Ubuntu Linux. The approach takes
inspiration from the Stellar project.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-02-23 19:54:49 +00:00
4775a9a77c service: support /var/lib/faasd/.docker/config.json
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-11 12:47:13 +00:00
e07186ed5b deploy: use reference.ParseNormalizedNamed()
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-11 12:47:13 +00:00
592f3d3cc0 Move to openfaas org
Closes: #36

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-02-06 09:36:18 +00:00
881f5171ee Remove two bad panic statements
Errors should be returned and handled in the caller.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-02-04 20:00:08 +00:00
970015ac85 Update order of code
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-02-04 20:00:08 +00:00
283e8ed2c1 Fix provider name to faasd-provider
Not sure how this got reverted / affected, but was wrong. The
name "faas-containerd" is gone and not in use.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-02-04 20:00:08 +00:00
d49011702b Refactor faasd and faas-containerd merge
The use of containerd and CNI functions has been refactored to reuse
the same codebase.

Added all network functionality to own directory and package. Removed
netlink and weave library in favor of using CNI plugin result files.

Rename containers handler to functions to clear-up functionality.

Signed-off-by: Carlos de Paula <me@carlosedp.com>
2020-02-04 10:12:43 +00:00
040b426a19 Set all permissions to 0644 vs a mixture
This appeared to prevent the provider's secret code from
creating files in its working directory. The patch makes all
code use the same permission.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-28 12:48:00 +00:00
5c48ac1a70 Add secrets support
Adds secrets support and binding of secrets at runtime to
functions. Files are written in plain-text to a 0644 permission
folder which can only be read by root and the containers
requesting the secret through the OpenFaaS API.

Tested by deploying an alpine function using "cat" as its
fprocess.

Happy to revisit at a later date and look into encryption at
rest. This should be on-par with using Kubernetes in its
default unencrypted state.

Fixes: #29

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-28 11:41:54 +00:00
969fc566e1 Fix unhandled range error and extend deletion timeout
Fixes a bug when attempting to access a non-existant IP from
GetIPfromPID called via the list API.

Renames the provider from faas-containerd

Updates function deletion grace period to 30s to prevent any
errors in the REST API during a long-running deletion.

Tested on Linux with the figlet function which by default takes
around 5s to delete due to its write_timeout value, the deletion
now blocks rather than throwing an error.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-26 21:03:37 +00:00
42e9c91ee9 Initial merge of faas-containerd
This patch completes part of the work in #20 by porting the code
for faas-containerd in-tree. When tested, I was able to deploy
and then remove figlet from the store on `x86_64`.

In a follow-up PR, duplication will be removed where possible
and consolidated with updated documentation.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-21 13:43:39 +00:00
95e278b29a Port networking to CNI
Replaced netns utility with CNI plugins to create a bridge network
and allow communication between containers with firewall plugin.

Overwrite the CNI config in case it exists. Allow updating the config
on new versions.

Signed-off-by: Carlos de Paula <me@carlosedp.com>
2020-01-10 18:30:43 +00:00
d135999d3b Set working directory for faasd / faas-containerd
* faasd writes secrets to wd + /secrets/*
* faas-containerd is passed a custom path to use to load the
secrets

Both services gain their work /run/ folders for temporary and
working files. Tested on RPi3 e2e with faasd install.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-05 12:00:43 +00:00
3068d03279 Set wd to /run/faasd
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-05 12:00:43 +00:00
098baba7cc Add unit test for proxy and shutdown channel
* Proxy has initial unit test and more can be added
* Shutdown channel and cancellation added for proper shutdown of
the proxy

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2020-01-03 12:06:53 +00:00
d785bebf4c Copy headers in both directions in proxy
* Issue was detected whilst testing 0.4.0 from @Waterdrips which
added basic auth, but the header was not being propagated.
* This code is tested in OpenFaaS already, but unit tests will
be added retrospectively.
* Proxy now reads the gateway URL via a channel instead of from
a file to make unit testing easier.

Basic auth now works as expected with faas-cli login / list.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-31 18:20:43 +00:00
300d8b082a Pass version from main
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-31 12:30:14 +00:00
17a5e2c625 Extract file for version command
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-31 12:18:27 +00:00
19a769b7da Update proxy print message
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-29 13:02:03 +00:00
48237e0b3c Don't follow redirects
Required for functioning proxy

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-29 12:59:39 +00:00
306313ed9a Proxy from faasd to gateway
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-29 12:46:32 +00:00
ff0cccf0dc Add proxy to faasd up
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-29 12:46:32 +00:00
f76432f60a Add use of template to Install command
The template name wasnt used, so the command gave an error saying
that no template was used.

Signed-off-by: Alistair Hey <alistair@heyal.co.uk>
2019-12-29 11:28:16 +00:00
38f26b213f Clear snapshot when container doesn't exist
This clears up a scenario where a container can be deleted but
its snapshot is not.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 20:01:01 +00:00
6c3fe813fd Extract PrepareImage
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 19:09:42 +00:00
13d28bd2db Extract Service struct
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 18:32:13 +00:00
5a28f3e231 Add error handling for when template not found
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-28 16:41:06 +00:00
3ee52c6ed7 Remove tasks and containers on SIGINT/SIGTERM
* Cleans-up and removes faasd containers/tasks when receiving
SIGINT/SIGTERM

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-24 10:09:56 +00:00
ad97b6db58 Add systemd utility package
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-24 09:12:34 +00:00
abc41d2108 Update queue-worker to print body
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-23 20:53:11 +00:00
9ba64783f9 Enable NATS, speed-up restart
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-12-23 20:43:00 +00:00