369 Commits

Author SHA1 Message Date
Alex Ellis (OpenFaaS Ltd)
cdfa01208d Bump versions
Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-08-22 15:48:23 +01:00
Alex Ellis
14677e0b9a Update alpine shipping versions
Alpine 3.10 is now availabe and is no-harm update.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-08-21 07:40:51 +01:00
Alex Ellis
e2321fb631 Update contact email
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-08-12 09:01:29 +01:00
Alex Ellis (OpenFaaS Ltd)
df97efafae Migrate away from requests package for Function structs
The function deployment and status structs have been moved away
into the faas-provider package.

Tested with a build, running tests, and CI.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
2019-08-05 12:58:30 +01:00
Alex Ellis
a156f26443 Create tag suffix using modified arch value
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-07-21 09:13:43 +01:00
Alex Ellis
c0ed8ac3f3 Add support for ARM64
This patch also removes the need for a separate ARM64 Dockerfile.
It has been tested on an Odroid-C2.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-07-20 10:43:35 +01:00
Alex Ellis
d3b3130586 Collapse Dockerfile.armhf into Dockerfile
This change which has been tested on armhf and x86_64 removes
the need for a separate Dockerfile for armhf.

CGO_ENABLED and GOARM etc are now passed as ARGs via build.sh.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-07-20 10:43:35 +01:00
Ivana Yovcheva (VMware)
2420b387b5 Add architecture to /system/info endpoint
With this change /system/info endpoint is going to give
information about the platform architecture it is running on
(arm64, armhf, x86_64)

Signed-off-by: Ivana Yovcheva (VMware) <iyovcheva@vmware.com>
2019-07-17 10:05:11 +01:00
Lucas Roesler
4c12c2e2f8 Add tests for loading the logs provider value from the env
**What**
- Ensure that we are loading the log provider url correctly, including
fallback to the function provider, when the value is set

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
d8a5952cfe Upgrade faas-provider to 0.9.2
Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
6868d8a3ce Document the new logs endpoint and env variable
**What**
- Document the new logs handler url env variable in the gateway readme
- Document the logs handler in the swagger docs
Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
74d08126b3 Handle unexpected statuscodes from the log provider
**What**
- Use NotImplemented when we get a 404 from the log provider.  When we
get a 200, process the log request, as normal.  For all other status
codes, return a sever error with a message stating that the
response was unexpected.  The message will contain the original status
code to assist with debugging

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
02ccccbe60 Remove duplicated code to remove hop headers
Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
c28175c4f6 Fix missing imports and tests
**What**
- Fix missing imports since my ide forgot to add the "time" package
- Update the tests to accept the handler timeout duration

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
6df51a3516 Use context timeout to cancel the log request
**What**
- Pass the writetimeout to the logs handler to set the context timeout
of the log stream.

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
db6628d1a5 Remove Hyjacker log handler
**What**
- Remove the hyjacker based logs handler implementation because it is
not needed

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
e07a61fd0c Reimplemnt the logs proxy without hijacking
**What**
- Create an alternative proxy implementation using CloseNotifier and
Flusher

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
00c734a136 Verify not goroutine leaks in the log proxy
**What**
- Add test to verify that the log proxy shutsdown correctly when the
client cancels
- Add test to verify that the log proxy shutsdown correctly when the
logs provider closes the connection

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Lucas Roesler
e7e91ecd15 Implement log proxy handler
**What**
- Implement log handler method that will hijack the connection and clear
timeouts to allow long lived streams
- Proxies requests to the logs provider and returns the response
unmodified

Signed-off-by: Lucas Roesler <roesler.lucas@gmail.com>
2019-07-06 10:42:46 +01:00
Alex Ellis
b275a2010c Update Dockerfile for ARM64
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-24 17:52:21 +01:00
Burton Rheutan
a6c9e3bb30 Remove extra customize button
The customize button was redundant to the custom tab
This could lead to confusion with the user experience,
so the additional button has been removed.

Signed-off-by: Burton Rheutan <rheutan7@gmail.com>
2019-06-17 10:20:22 +01:00
Burton Rheutan
a987d2147b Add additional fields to the UI
Adding additional inputs for Environment variables, Secrets,
Labels, and annotations so that more advanced functions can
be deployed through the UI.

This also allows more advanced functions to be provided via
the store. Allowing modification to secret values, and
environment variables which will allow functions like a
Slack Bot function to be deployed through the store adn
configured with an environment variable

Signed-off-by: Burton Rheutan <rheutan7@gmail.com>
2019-06-16 14:58:20 +01:00
Alex Ellis
6481b683f2 Add ca-certs to multi-arch gateways
Adds package to armhf / arm64 for use with plugins.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-14 19:19:15 +01:00
Alex Ellis
e3c976a428 Fix error handling for ExternalAuth
This corrects an issue where the error body was being hidden
for the external auth handler. It also adds the ca-certs into
the runtime Docker image for when the gateway is calling an
external plugin exposed over HTTPS.

Tested with OAuth2 plugin.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-14 19:17:25 +01:00
Alex Ellis
e3b77514d0 Add error handling to basic auth injector
Fixes a problem where basic auth was disabled and a nill pointer
was hit, causing a panic.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-09 20:08:39 +01:00
Alex Ellis
d2965df9f2 Remove un-used RoutelessProxy
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-09 20:08:39 +01:00
Alex Ellis
1cf030da48 Differentiate external service auth from user auth
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-09 20:08:39 +01:00
Alex Ellis
0758e484f7 Switch to external auth
This commit moves the OpenFaaS gateway from using in-process
basic-auth for everything to use an external auth URL instead.

When auth is not enable, this functionality is not added to the
handlers and behaves as before. When enabled, the configured
plugin with authenticate requests.

Tested on Docker Swarm with positive and negative tests.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-09 20:08:39 +01:00
Alex Ellis
701708fe0e Update faas-provider version
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-08 10:16:30 +01:00
Alex Ellis
d6b3847fbd Add body from basic auth plugin.
Fixes issue by adding unit test to make sure the body from
the plugin is written correctly and proxied to the client.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-07 10:11:40 +01:00
Alex Ellis
ef811783fb Pass headers back to client from auth plugin
Fix for external auth wrapper handler. Written by introducing
a broken unit test. Whenever the auth plugin returns a request
as not authorized, we must pass back any headers set by the
plugin.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-07 09:46:30 +01:00
Alex Ellis
6beca8f59b Pass headers when using external auth
Fixes issue found in e2e testing where the headers were not
being passed to the basic-auth-plugin. This change makes sure
the upstream check gets all headers copied in before making
the call.

Tested with negative unit tests before writing fix.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-07 09:37:03 +01:00
Alex Ellis
7be07e2668 Fix broken test
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-05 18:28:26 +01:00
Alex Ellis
c9b5e5f146 Move to use WithTimeout instead of WithDeadline for context
These two functions are effectively the same, with the former
being a wrapper for the later.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-05 18:13:49 +01:00
Alex Ellis
35508ac70b Add explicit deadline for auth request
An explicit timeout is passed to the handler and a new unit test
proves that the functionality is in place. A additional return
statement was needed in the handler as pointed out by
@stefanprodan.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-05 18:13:49 +01:00
Alex Ellis
a66097a9f9 Add config options for auth proxy
Adds two new environment variables and unit tests to validate
positive and default use-cases.

auth_proxy_url
auth_proxy_pass_body

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-05 18:13:49 +01:00
Alex Ellis
5b2a037e7e Add external auth handler
This commit adds an external auth handler which can be used to
wrap existing handlers, so that they delegate their requests
to an upstream URL before allowing a request to pass through
to an upstream API.

New handler tested with unit tests.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-06-05 18:13:49 +01:00
Rishabh Gupta
8c896d92ee Fixed arm64 dockerfile
Signed-off-by: Rishabh Gupta <r.g.gupta@outlook.com>
2019-04-14 16:51:22 +01:00
Rishabh Gupta
7b09992565 Using functions.json instead of store-arch.json
Signed-off-by: Rishabh Gupta <r.g.gupta@outlook.com>
2019-04-14 16:51:22 +01:00
Alex Ellis
b26b95cab5 Update Dockerfiles
There is a utility script which can be used for installing
license-check. This is added to the gateway Dockerfiles.

Tested in Docker for Mac.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-04-11 20:57:17 +01:00
Alex Ellis
485a33e4fb Quote source of hop headers
Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-04-06 20:45:04 +01:00
Alex Ellis
78c127619e Remove hop headers
Requested by @LucasRoesler - removes headers detailed in HTTP
spec which are not supposed to be forwarded by proxies or
gateways.

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-04-06 20:45:04 +01:00
Vivek Singh
b87ecde60f Move /healthz handler to handlers package
Signed-off-by: Vivek Singh <vivekkmr45@yahoo.in>
2019-03-09 17:28:46 +00:00
Vivek Singh
40dbede065 Reduce timeout value metrics server to 5 second
Signed-off-by: Vivek Singh <vivekkmr45@yahoo.in>
2019-03-09 17:28:46 +00:00
Vivek Singh
54eda5ec0d Run metrics server in a separate method
This commit adds changes to run metrics server in a separate method and
also removed port 8082 from exposed port a/c to review comments.

It also uses a smaller static timeout value for new server.

Signed-off-by: Vivek Singh <vivekkmr45@yahoo.in>
2019-03-09 17:28:46 +00:00
Vivek Singh
90ddd56985 Add new HTTPServer to serve /metrics on port 8082
This commit runs a new HTTPServer on port 8082 in a goroutine to serve
/metrics endpoint on a different port.

This also update the configurations and compose files.

Fixes: #1081

Signed-off-by: Vivek Singh <vivekkmr45@yahoo.in>
2019-03-09 17:28:46 +00:00
Alex Ellis
c394b09ae6 Bump to nats-queue-worker 0.7.0
Includes fix for reconnection bug to NATS Streaming

Signed-off-by: Alex Ellis <alexellis2@gmail.com>
2019-02-12 17:46:00 +00:00
Alex Ellis (VMware)
299e5a5933 Read config values from environment for max_conns tuning
- max_conns / idle / per host are now read from env-vars and have
defaults set to 1024 for both values
- logging / metrics are collected in the client transaction
rather than via defer (this may impact throughput)
- function cache moved to use RWMutex to try to improve latency
around locking when updating cache
- logging message added to show latency in running GetReplicas
because this was observed to increase in a linear fashion under
high concurrency
- changes tested against 3-node bare-metal 1.13 K8s cluster
with kubeadm

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-02-04 11:50:25 +00:00
Alex Ellis (VMware)
52c27e227a Tune HTTP client for concurrency
- due to what appears to be a frequent issue with the Go HTTP
client some tweaks were needed to the HTTP client used for
reverse proxying to prevent CoreDNS from rejecting connections.

The following PRs / commits implement similar changes in
Prometheus and Minio.

https://github.com/prometheus/prometheus/pull/3592
https://github.com/minio/minio/pull/5860

Under a 3-node (1-master) kubeadm cluster running on bare
metal with Ubuntu 18.04 I was able to send 100k requests
with 1000 being concurrent with no errors being returned
by hey.

```
hey -n 100000 -c 1000 -m=POST -d="hi" \
  http://192.168.0.26:31112/function/go-echo
```

The go-echo function is based upon the golang-http
template in the function store using the of-watchdog.

Signed-off-by: Alex Ellis (VMware) <alexellis2@gmail.com>
2019-02-04 11:50:25 +00:00
Radoslav Dimitrov
c5122279c9 Fix unit test fail due to race condition #1063
Signed-off-by: Radoslav Dimitrov <dimitrovr@vmware.com>
2019-01-30 10:28:13 +00:00