# Troubleshooting guide

## Timeouts

Default timeouts are configured at the HTTP level and must be set both on the gateway and the function.

> Note: all distributed systems need a maximum timeout value to be configured for work. This means that work cannot be unbounded.

### Timeouts - Your function

You can also enforce a hard-timeout for your function with the `hard_timeout` environmental variable.

For watchdog configuration see the [README](https://github.com/openfaas/faas/tree/master/watchdog).

The best way to set the timeout is in the YAML file generated by the `faas-cli`.

Example Go app that sleeps for (10 seconds):

```
provider:
  name: faas
  gateway: http://localhost:8080

functions:
  sleepygo:
    lang: go
    handler: ./sleepygo
    image: alexellis2/sleeps-for-10-seconds
    environment:
        read_timeout: 20s
        write_timeout: 20s
```

handler.go

```
package function

...

func Handle(req []byte) string {
	time.Sleep(time.Second * 10)
	return fmt.Sprintf("Hello, Go. You said: %s", string(req))
}
```

### Timeouts - Gateway

For the gateway set the following environmental variables:

```
            read_timeout:  "25s"        # Maximum time to read HTTP request
            write_timeout: "25s"        # Maximum time to write HTTP response
            upstream_timeout: "20s"     # Maximum duration of upstream function call
```

> Note: The value for `upstream_timeout` should be slightly less than `read_timeout` and `write_timeout`

### Timeouts - Function provider

When using a gateway version older than `0.7.8` a timeout matching the gateway should be set for the `faas-swarm` or `faas-netes` controller.

```
read_timeout: 25s
write_timeout: 25s
```

### Timeouts - Asynchronous invocations

For asynchronous invocations of functions a separate timeout can be configured at the `queue-worker` level in the `ack_timeout` environmental variable.

If the `ack_timeout` is exceeded the task will not be acknowledge and the queue system will retry the invocation.

## Function execution logs

By default the functions will not log out the result, but just show how long the process took to run and the length of the result in bytes.

```
$ echo test this | faas invoke json-hook -g localhost:31112
Received JSON webook. Elements: 10

$ kubectl logs deploy/json-hook -n openfaas-fn
2018/01/28 20:47:21 Writing lock-file to: /tmp/.lock
2018/01/28 20:47:27 Forking fprocess.
2018/01/28 20:47:27 Wrote 35 Bytes - Duration: 0.001844 seconds
```

If you want to see the result of a function in the function's logs then deploy it with the `write_debug` environmental variable set to `true`.

For example:

```
provider:
  name: faas
  gateway: http://localhost:8080

functions:
  json-hook:
    lang: go
    handler: ./json-hook
    image: json-hook
    environment:
      write_debug: true
```

Now you'll see logs like this:

```
$ echo test this | faas invoke json-hook -g localhost:31112
Received JSON webook. Elements: 10

$ kubectl logs deploy/json-hook -n openfaas-fn
2018/01/28 20:50:27 Writing lock-file to: /tmp/.lock
2018/01/28 20:50:35 Forking fprocess.
2018/01/28 20:50:35 Query  
2018/01/28 20:50:35 Path  /function/json-hook
Received JSON webook. Elements: 10
2018/01/28 20:50:35 Duration: 0.001857 seconds
```

You can then find the logs of the function using Docker Swarm or Kubernetes as listed in the section below.

## Healthcheck

Most problems reported via GitHub or Slack stem from a configuration problem or issue with a function. Here is a checklist of things you can try before digging deeper:

Checklist:
* [ ] All core services are deployed: i.e. gateway
* [ ] Check functions are deployed and started
* [ ] Check request isn't timing out at the gateway or the function level

## CLI unresponsive - localhost vs 127.0.0.1

On certain Linux distributions the name `localhost` maps to an IPv6 alias meaning that the CLI may hang. In these circumstances you have two options:

1. Use the `-g` or `--gateway` argument with `127.0.0.1`

This forces IPv4.

2. Edit the `/etc/hosts` file on your machine and remove the IPv6 alias for localhost.

# Troubleshooting Swarm or Kubernetes

## Docker Swarm

### List all functions

```
$ docker service ls
```

You are looking for 1/1 for the replica count of each service listed.

### Find a function's logs

```
$ docker service logs --tail 100 FUNCTION
```

### Find out if a function failed to start

```
$ docker service ps --no-trunc=true FUNCTION
```

### Stop and remove OpenFaaS

```
$ docker stack rm func
```

If you have additional services / functions remove the remaining ones like this:

```
$ docker service ls -q | xargs docker service rm
```

*Use with caution*

## Kubernetes

If you have deployed OpenFaaS to the recommended namespaces then functions are in the `openfaas-fn` namespace and the core services are in the `openfaas` namespace. The `-n` flag to `kubectl` sets the namespace to look at.

### List OpenFaaS services

```
$ kubectl get deploy -n openfaas
```

### List all functions

```
$ kubectl get deploy -n openfaas-fn
```

### Find a function's logs

```
$ kubectl logs -n openfaas-fn deploy/FUNCTION_NAME
```

### Find out if a function failed to start

```
$ kubectl describe -n openfaas-fn deploy/FUNCTION_NAME
```

### Remove the OpenFaaS deployment

From within the `faas-netes` folder:

```
$ kubectl delete -f namespaces.yml,./yaml/
```

# Watchdog

## Debug your function without deploying it

Here's an example of how you can deploy a function without using an orchestrator and the API gateeway. It is especially useful for testing:

```
$ docker run --name debug-alpine \
  -p 8081:8080 -ti functions/alpine:latest sh
# fprocess=date fwatchdog &
```

Now you can access the function with one of the supported HTTP methods such as GET/POST etc:

```
$ curl -4 localhost:8081
```

## Edit your function without rebuilding it

You can bind-mount code straight into your function and work with it locally, until you are ready to re-build. This is a common flow with containers, but should be used sparingly.

Within the CLI directory for instance:

Build the samples:

```
$ git clone https://github.com/openfaas/faas-cli && \
  cd faas-cli
$ faas-cli -action build -f ./samples.yml
```

Now work with the Python-hello sample, with the code mounted live:

```
$ docker run -v `pwd`/sample/url-ping/:/root/function/ \
  --name debug-alpine -p 8081:8080 -ti alexellis/faas-url-ping sh
$ touch ./function/__init__.py
# fwatchdog
```

Now you can start editing the code in the sample/url-ping folder and it will reload live for every request.

```
$ curl localhost:8081 -d "https://www.google.com"
Handle this -> https://www.google.com
https://www.google.com => 200
```

Now you can edit handler.py and you'll see the change immediately:

```
$ echo "def handle(req):" > sample/url-ping/handler.py
$ echo '    print("Nothing to see here")' >> sample/url-ping/handler.py
$ curl localhost:8081 -d "https://www.google.com"
Nothing to see here
```
