mirror of
https://github.com/openfaas/faas.git
synced 2025-06-10 09:16:48 +00:00
264 lines
6.3 KiB
Markdown
264 lines
6.3 KiB
Markdown
# Troubleshooting guide
|
|
|
|
## CLI unresponsive - localhost vs 127.0.0.1
|
|
|
|
On certain Linux distributions the name `localhost` maps to an IPv6 alias meaning that the CLI may hang. In these circumstances you have two options:
|
|
|
|
1. Use the `-g` or `--gateway` argument with `127.0.0.1`
|
|
|
|
This forces IPv4.
|
|
|
|
2. Edit the `/etc/hosts` file on your machine and remove the IPv6 alias for localhost.
|
|
|
|
## Timeouts
|
|
|
|
Default timeouts are configured at the HTTP level and must be set both on the gateway and the function.
|
|
|
|
> Note: all distributed systems need a maximum timeout value to be configured for work. This means that work cannot be unbounded.
|
|
|
|
### Timeouts - Your function
|
|
|
|
You can also enforce a hard-timeout for your function with the `hard_timeout` environmental variable.
|
|
|
|
For watchdog configuration see the [README](https://github.com/openfaas/faas/tree/master/watchdog).
|
|
|
|
The best way to set the timeout is in the YAML file generated by the `faas-cli`.
|
|
|
|
Example Go app that sleeps for (10 seconds):
|
|
|
|
```
|
|
provider:
|
|
name: faas
|
|
gateway: http://localhost:8080
|
|
|
|
functions:
|
|
sleepygo:
|
|
lang: go
|
|
handler: ./sleepygo
|
|
image: alexellis2/sleeps-for-10-seconds
|
|
environment:
|
|
read_timeout: 20
|
|
write_timeout: 20
|
|
```
|
|
|
|
handler.go
|
|
|
|
```
|
|
package function
|
|
|
|
...
|
|
|
|
func Handle(req []byte) string {
|
|
time.Sleep(time.Second * 10)
|
|
return fmt.Sprintf("Hello, Go. You said: %s", string(req))
|
|
}
|
|
```
|
|
|
|
### Timeouts - Gateway
|
|
|
|
For the gateway set the following environmental variables:
|
|
|
|
```
|
|
read_timeout: 30
|
|
write_timeout: 30
|
|
```
|
|
|
|
The default for both is "8" - seconds. In the example above "30" means 30 seconds.
|
|
|
|
### Timeouts - Function provider
|
|
|
|
If on Kubernetes and Swarm you should set a matching timeout for the faas-netesd or faas-swarm controller matching that of the gateway.
|
|
|
|
```
|
|
read_timeout: 30
|
|
write_timeout: 30
|
|
```
|
|
|
|
### Timeouts - Asynchronous invocations
|
|
|
|
For asynchronous invocations of functions a separate timeout can be configured at the `queue-worker` level in the `ack_timeout` environmental variable.
|
|
|
|
If the `ack_timeout` is exceeded the task will not be acknowledge and the queue system will retry the invocation.
|
|
|
|
## Function execution logs
|
|
|
|
By default the functions will not log out the result, but just show how long the process took to run and the length of the result in bytes.
|
|
|
|
```
|
|
$ echo test this | faas invoke json-hook -g localhost:31112
|
|
Received JSON webook. Elements: 10
|
|
|
|
$ kubectl logs deploy/json-hook -n openfaas-fn
|
|
2018/01/28 20:47:21 Writing lock-file to: /tmp/.lock
|
|
2018/01/28 20:47:27 Forking fprocess.
|
|
2018/01/28 20:47:27 Wrote 35 Bytes - Duration: 0.001844 seconds
|
|
```
|
|
|
|
If you want to see the result of a function in the function's logs then deploy it with the `write_debug` environmental variable set to `true`.
|
|
|
|
For example:
|
|
|
|
```
|
|
provider:
|
|
name: faas
|
|
gateway: http://localhost:8080
|
|
|
|
functions:
|
|
json-hook:
|
|
lang: go
|
|
handler: ./json-hook
|
|
image: json-hook
|
|
environment:
|
|
write_debug: true
|
|
```
|
|
|
|
Now you'll see logs like this:
|
|
|
|
```
|
|
$ echo test this | faas invoke json-hook -g localhost:31112
|
|
Received JSON webook. Elements: 10
|
|
|
|
$ kubectl logs deploy/json-hook -n openfaas-fn
|
|
2018/01/28 20:50:27 Writing lock-file to: /tmp/.lock
|
|
2018/01/28 20:50:35 Forking fprocess.
|
|
2018/01/28 20:50:35 Query
|
|
2018/01/28 20:50:35 Path /function/json-hook
|
|
Received JSON webook. Elements: 10
|
|
2018/01/28 20:50:35 Duration: 0.001857 seconds
|
|
```
|
|
|
|
You can then find the logs of the function using Docker Swarm or Kubernetes as listed in the section below.
|
|
|
|
## Healthcheck
|
|
|
|
Most problems reported via GitHub or Slack stem from a configuration problem or issue with a function. Here is a checklist of things you can try before digging deeper:
|
|
|
|
Checklist:
|
|
* [ ] All core services are deployed: i.e. gateway
|
|
* [ ] Check functions are deployed and started
|
|
* [ ] Check request isn't timing out at the gateway or the function level
|
|
|
|
# Troubleshooting Swarm or Kubernetes
|
|
|
|
## Docker Swarm
|
|
|
|
### List all functions
|
|
|
|
```
|
|
$ docker service ls
|
|
```
|
|
|
|
You are looking for 1/1 for the replica count of each service listed.
|
|
|
|
### Find a function's logs
|
|
|
|
```
|
|
$ docker service logs --tail 100 FUNCTION
|
|
```
|
|
|
|
### Find out if a function failed to start
|
|
|
|
```
|
|
$ docker service ps --no-trunc=true FUNCTION
|
|
```
|
|
|
|
### Stop and remove OpenFaaS
|
|
|
|
```
|
|
$ docker stack rm func
|
|
```
|
|
|
|
If you have additional services / functions remove the remaining ones like this:
|
|
|
|
```
|
|
$ docker service ls -q | xargs docker service rm
|
|
```
|
|
|
|
*Use with caution*
|
|
|
|
## Kubernetes
|
|
|
|
### List all functions
|
|
|
|
```
|
|
$ kubectl get deploy
|
|
```
|
|
|
|
### Find a function's logs
|
|
|
|
```
|
|
$ kubectl logs deploy/FUNCTION
|
|
```
|
|
|
|
### Find out if a function failed to start
|
|
|
|
```
|
|
$ kubectl describe deploy/FUNCTION
|
|
```
|
|
|
|
### Remove the OpenFaaS deployment
|
|
|
|
```
|
|
$ git clone https://github.com/openfaas/faas-netes/ && \
|
|
cd faas-netes && \
|
|
kubectl delete -f ./yaml/
|
|
```
|
|
|
|
# Watchdog
|
|
|
|
## Debug your function without deploying it
|
|
|
|
Here's an example of how you can deploy a function without using an orchestrator and the API gateeway. It is especially useful for testing:
|
|
|
|
```
|
|
$ docker run --name debug-alpine \
|
|
-p 8081:8080 -ti functions/alpine:latest sh
|
|
# fprocess=date fwatchdog &
|
|
```
|
|
|
|
Now you can access the function with one of the supported HTTP methods such as GET/POST etc:
|
|
|
|
```
|
|
$ curl -4 localhost:8081
|
|
```
|
|
|
|
## Edit your function without rebuilding it
|
|
|
|
You can bind-mount code straight into your function and work with it locally, until you are ready to re-build. This is a common flow with containers, but should be used sparingly.
|
|
|
|
Within the CLI directory for instance:
|
|
|
|
Build the samples:
|
|
|
|
```
|
|
$ git clone https://github.com/openfaas/faas-cli && \
|
|
cd faas-cli
|
|
$ faas-cli -action build -f ./samples.yml
|
|
```
|
|
|
|
Now work with the Python-hello sample, with the code mounted live:
|
|
|
|
```
|
|
$ docker run -v `pwd`/sample/url-ping/:/root/function/ \
|
|
--name debug-alpine -p 8081:8080 -ti alexellis/faas-url-ping sh
|
|
$ touch ./function/__init__.py
|
|
# fwatchdog
|
|
```
|
|
|
|
Now you can start editing the code in the sample/url-ping folder and it will reload live for every request.
|
|
|
|
```
|
|
$ curl localhost:8081 -d "https://www.google.com"
|
|
Handle this -> https://www.google.com
|
|
https://www.google.com => 200
|
|
```
|
|
|
|
Now you can edit handler.py and you'll see the change immediately:
|
|
|
|
```
|
|
$ echo "def handle(req):" > sample/url-ping/handler.py
|
|
$ echo ' print("Nothing to see here")' >> sample/url-ping/handler.py
|
|
$ curl localhost:8081 -d "https://www.google.com"
|
|
Nothing to see here
|
|
```
|