454 Commits

Author SHA1 Message Date
Lucas Roesler
db6628d1a5 Remove Hyjacker log handler
**What**
- Remove the hyjacker based logs handler implementation because it is
not needed

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
e07a61fd0c Reimplemnt the logs proxy without hijacking
**What**
- Create an alternative proxy implementation using CloseNotifier and
Flusher

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
00c734a136 Verify not goroutine leaks in the log proxy
**What**
- Add test to verify that the log proxy shutsdown correctly when the
client cancels
- Add test to verify that the log proxy shutsdown correctly when the
logs provider closes the connection

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
e7e91ecd15 Implement log proxy handler
**What**
- Implement log handler method that will hijack the connection and clear
timeouts to allow long lived streams
- Proxies requests to the logs provider and returns the response
unmodified

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Alex Ellis
b275a2010c Update Dockerfile for ARM64
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-24 17:52:21 +01:00
Burton Rheutan
a6c9e3bb30 Remove extra customize button
The customize button was redundant to the custom tab
This could lead to confusion with the user experience,
so the additional button has been removed.

Signed-off-by: Burton Rheutan <rheutan7@gmail.com>
2019-06-17 10:20:22 +01:00
Burton Rheutan
a987d2147b Add additional fields to the UI
Adding additional inputs for Environment variables, Secrets,
Labels, and annotations so that more advanced functions can
be deployed through the UI.

This also allows more advanced functions to be provided via
the store. Allowing modification to secret values, and
environment variables which will allow functions like a
Slack Bot function to be deployed through the store adn
configured with an environment variable

Signed-off-by: Burton Rheutan <rheutan7@gmail.com>
2019-06-16 14:58:20 +01:00
Alex Ellis
6481b683f2 Add ca-certs to multi-arch gateways
Adds package to armhf / arm64 for use with plugins.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-14 19:19:15 +01:00
Alex Ellis
e3c976a428 Fix error handling for ExternalAuth
This corrects an issue where the error body was being hidden
for the external auth handler. It also adds the ca-certs into
the runtime Docker image for when the gateway is calling an
external plugin exposed over HTTPS.

Tested with OAuth2 plugin.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-14 19:17:25 +01:00
Alex Ellis
e3b77514d0 Add error handling to basic auth injector
Fixes a problem where basic auth was disabled and a nill pointer
was hit, causing a panic.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-09 20:08:39 +01:00
Alex Ellis
d2965df9f2 Remove un-used RoutelessProxy
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-09 20:08:39 +01:00
Alex Ellis
1cf030da48 Differentiate external service auth from user auth
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-09 20:08:39 +01:00
Alex Ellis
0758e484f7 Switch to external auth
This commit moves the OpenFaaS gateway from using in-process
basic-auth for everything to use an external auth URL instead.

When auth is not enable, this functionality is not added to the
handlers and behaves as before. When enabled, the configured
plugin with authenticate requests.

Tested on Docker Swarm with positive and negative tests.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-09 20:08:39 +01:00
Alex Ellis
701708fe0e Update faas-provider version
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-08 10:16:30 +01:00
Alex Ellis
d6b3847fbd Add body from basic auth plugin.
Fixes issue by adding unit test to make sure the body from
the plugin is written correctly and proxied to the client.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-07 10:11:40 +01:00
Alex Ellis
ef811783fb Pass headers back to client from auth plugin
Fix for external auth wrapper handler. Written by introducing
a broken unit test. Whenever the auth plugin returns a request
as not authorized, we must pass back any headers set by the
plugin.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-07 09:46:30 +01:00
Alex Ellis
6beca8f59b Pass headers when using external auth
Fixes issue found in e2e testing where the headers were not
being passed to the basic-auth-plugin. This change makes sure
the upstream check gets all headers copied in before making
the call.

Tested with negative unit tests before writing fix.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-07 09:37:03 +01:00
Alex Ellis
7be07e2668 Fix broken test
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-05 18:28:26 +01:00
Alex Ellis
c9b5e5f146 Move to use WithTimeout instead of WithDeadline for context
These two functions are effectively the same, with the former
being a wrapper for the later.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-05 18:13:49 +01:00
Alex Ellis
35508ac70b Add explicit deadline for auth request
An explicit timeout is passed to the handler and a new unit test
proves that the functionality is in place. A additional return
statement was needed in the handler as pointed out by
@stefanprodan.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-05 18:13:49 +01:00
Alex Ellis
a66097a9f9 Add config options for auth proxy
Adds two new environment variables and unit tests to validate
positive and default use-cases.

auth_proxy_url
auth_proxy_pass_body

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-05 18:13:49 +01:00
Alex Ellis
5b2a037e7e Add external auth handler
This commit adds an external auth handler which can be used to
wrap existing handlers, so that they delegate their requests
to an upstream URL before allowing a request to pass through
to an upstream API.

New handler tested with unit tests.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-05 18:13:49 +01:00
Rishabh Gupta
8c896d92ee Fixed arm64 dockerfile
Signed-off-by: Rishabh Gupta <r.g.gupta@outlook.com>
2019-04-14 16:51:22 +01:00
Rishabh Gupta
7b09992565 Using functions.json instead of store-arch.json
Signed-off-by: Rishabh Gupta <r.g.gupta@outlook.com>
2019-04-14 16:51:22 +01:00
Alex Ellis
b26b95cab5 Update Dockerfiles
There is a utility script which can be used for installing
license-check. This is added to the gateway Dockerfiles.

Tested in Docker for Mac.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-04-11 20:57:17 +01:00
Alex Ellis
485a33e4fb Quote source of hop headers
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-04-06 20:45:04 +01:00
Alex Ellis
78c127619e Remove hop headers
Requested by @LucasRoesler - removes headers detailed in HTTP
spec which are not supposed to be forwarded by proxies or
gateways.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-04-06 20:45:04 +01:00
Vivek Singh
b87ecde60f Move /healthz handler to handlers package
Signed-off-by: Vivek Singh <vivekkmr45@yahoo.in>
2019-03-09 17:28:46 +00:00
Vivek Singh
40dbede065 Reduce timeout value metrics server to 5 second
Signed-off-by: Vivek Singh <vivekkmr45@yahoo.in>
2019-03-09 17:28:46 +00:00
Vivek Singh
54eda5ec0d Run metrics server in a separate method
This commit adds changes to run metrics server in a separate method and
also removed port 8082 from exposed port a/c to review comments.

It also uses a smaller static timeout value for new server.

Signed-off-by: Vivek Singh <vivekkmr45@yahoo.in>
2019-03-09 17:28:46 +00:00
Vivek Singh
90ddd56985 Add new HTTPServer to serve /metrics on port 8082
This commit runs a new HTTPServer on port 8082 in a goroutine to serve
/metrics endpoint on a different port.

This also update the configurations and compose files.

Fixes: #1081

Signed-off-by: Vivek Singh <vivekkmr45@yahoo.in>
2019-03-09 17:28:46 +00:00
Alex Ellis
c394b09ae6 Bump to nats-queue-worker 0.7.0
Includes fix for reconnection bug to NATS Streaming

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-02-12 17:46:00 +00:00
Alex Ellis (VMware)
299e5a5933 Read config values from environment for max_conns tuning
- max_conns / idle / per host are now read from env-vars and have
defaults set to 1024 for both values
- logging / metrics are collected in the client transaction
rather than via defer (this may impact throughput)
- function cache moved to use RWMutex to try to improve latency
around locking when updating cache
- logging message added to show latency in running GetReplicas
because this was observed to increase in a linear fashion under
high concurrency
- changes tested against 3-node bare-metal 1.13 K8s cluster
with kubeadm

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-02-04 11:50:25 +00:00
Alex Ellis (VMware)
52c27e227a Tune HTTP client for concurrency
- due to what appears to be a frequent issue with the Go HTTP
client some tweaks were needed to the HTTP client used for
reverse proxying to prevent CoreDNS from rejecting connections.

The following PRs / commits implement similar changes in
Prometheus and Minio.

https://github.com/prometheus/prometheus/pull/3592
https://github.com/minio/minio/pull/5860

Under a 3-node (1-master) kubeadm cluster running on bare
metal with Ubuntu 18.04 I was able to send 100k requests
with 1000 being concurrent with no errors being returned
by hey.

```
hey -n 100000 -c 1000 -m=POST -d="hi" \
  http://192.168.0.26:31112/function/go-echo
```

The go-echo function is based upon the golang-http
template in the function store using the of-watchdog.

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-02-04 11:50:25 +00:00
Radoslav Dimitrov
c5122279c9 Fix unit test fail due to race condition #1063
Signed-off-by: Radoslav Dimitrov <dimitrovr@vmware.com>
2019-01-30 10:28:13 +00:00
Alex Ellis (VMware)
b4a550327d Re-vendor queue-worker publisher for reconnect
- re-vendor queue-worker for publisher via 0.6.0
- bump queue-worker version to 0.6.0 in docker-compose.yml for
AMD64
- use new naming for NATS of nats -> NATS in variables where
required
- add default reconnect of 60 times, 2 seconds apart.

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-01-29 15:15:48 +00:00
Lucas Roesler
f61735b155 Add basic auth to the system alert endpoint
**What**
- Protect the `/system/alert` endpoint when basic auth is enabled
- Update the alert manager config to send the basic auth credentials
- Bump the gateway version

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-01-24 17:45:33 +00:00
Alex Ellis (VMware)
ec185bad67 Fix label order for http_requests_total
- the order of http_requests_total was shown to be incorrect in
testing. This fixes the order as per
http_request_duration_seconds.

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-01-24 09:46:14 +00:00
Alex Ellis (VMware)
a26d350376 Allow unicode in service paths
- according to discussion in #1013 all unicode characters are
valid label values - this commit allows the original path to be
retained.

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-01-24 09:12:46 +00:00
Alex Ellis (VMware)
67c9a71686 Add unit tests for MakeNotifierWrapper
- fixes issue where result was assigned to value rather than
to pointer reference.

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-01-24 09:12:46 +00:00
Alex Ellis (VMware)
f7cf7a6496 Split out notifiers
- splits out notifiers and writes status for async handler

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-01-24 09:12:46 +00:00
Alex Ellis (VMware)
fca32a0e79 Instrument async handlers
- instruments async handler for report and for queueing async
requests
- make MustRegister only ever run once to prevent sync issues

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-01-24 09:12:46 +00:00
Alex Ellis (VMware)
5a1bdcdb91 Add instrumentation to the alert handler
Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-01-24 09:12:46 +00:00
Alex Ellis (VMware)
e9cf708cb5 Bump Prometheus client version
- updates the Prometheus go client version and switches to the
promhttp handler to avoid conflicts with the new system-level
metrics.

Tested with Docker Swarm locally - no conflicts and new metrics
were gathered.

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-01-24 09:12:46 +00:00
Alex Ellis (VMware)
64a3f4e495 Instrument system calls
Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-01-24 09:12:46 +00:00
Alex Ellis (VMware)
1cc767e898 Add service RED metrics definitions
Partially fixes #532 by introducing two metrics that are
supported by Kubernetes HPAv2 and RED metrics-style
dashboards.

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-01-24 09:12:46 +00:00
Richard Gee
0f5ca96bbe Add build-args to Dockerfile.armhf
When the /system/info endpoint was expanded to include information about the gateway a number of build-args were added to the main Dockerfile.  These changes were not mirrored in Dockerfile.armhf, which resulted in nil attributes and an ugly error when running `faas version` against an armhf gateway.

This change carries the changes made to Dockerfile through to Dockerfile.armhf.  As well as the build-args which fix the identified issue the license check has also been added at the latest release 0.2.3, as a armhf build has been made available.  Further changes are to introduce the app user and moving the binary location from /root/ to /home/app/

Signed-off-by: Richard Gee <richard@technologee.co.uk>
2019-01-20 10:00:02 +00:00
Burton Rheutan
988c855163 Gateway UI - validate manual input
This change validates manual input to the gateway UI when deploying
new functions. This is to prevent poor user experience when attempting
to deploy a function manually from the UI.

The validation check on the function name is the same pattern that
is used in the CLI to ensure that when the deploy button is pressed,
the function will not fail validation.

Signed-off-by: Burton Rheutan <rheutan7@gmail.com>
2019-01-19 11:00:59 +00:00
Radoslav Dimitrov
41b452849c Add a consistent ARM64 image build process
Signed-off-by: Radoslav Dimitrov <rdimitrow@gmail.com>
2019-01-16 09:09:41 +00:00
Alex Ellis (VMware)
a65df4795b Update swagger for missing secret definitions
- added secret definition and removed types used previously

Remove structs for secrets

- after discussion on PR the core contributors decided we just
want simple CRUD with the Secret type.

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-01-04 16:51:01 +00:00